What you’re experiencing is not uncommon, and I think your instincts are good. Too many writers let a book spiral out of their control, particularly early in their writing career.
What I suggest are little accents that evoke a much larger world, but which you don’t dwell upon. Spend a day or two brainstorming small “Tells” of various cultures and events outside the main story–a specific kind of clothing, a specific way of speaking, or an event that people commonly refer to. Don’t explain these things, but attach them in your notes to a culture or event, and give them a one line description. Don’t worry about filling that out, just put them in as seeds if you want them.
Again, don’t dwell on them. Make them curious, interesting, but don’t let them steal the show. Focus on your story inside the city, with your main characters. If you do this right, it will give the feeling that a huge world is out there–and the reader will be comfortable knowing it’s there, but will have their attention focused on your main story.