Is it legal to live in your detched garage?

we have turned our garage into a back room. there is no bathroom in it but we took the garage doors out and made it look like a little house. its really nice inside but anyways one of our neighbors has one that was not fixed up like ours and the city came and said that it was unlivable and made the couple that was living there move out. now they are threatning to tell the city about our room and the couple that lives in ours. we live in fresno ca if that helps