Yes, that's a clean out and you need or replace the plug or use a clean out cover and plug the whole on the center (if you don't your wall will be worse with the water coming out)
That is sewer juice. You sure don't want that leaking in your closet. Try removing the cap, and give the threads a good cleaning. Re-install using some vaseline on the threads or if the threads are damaged, you may need to buy a new cap.