I used to stay in little India. Plenty of cheap hotels and places to eat there. There are some budget hotel chains worth checking out also. Such as:
Hotel 81
Singapore is easy to get around so I wouldn't be too bothered even if you are a bit far from the embassy. I would suggest using an agent anyway. Ismael for example will send a courier to your hotel to pick up and drop off your documents. The extra money you pay for using the agent could still be cheaper than doing it yourself and having to spend longer than required in Singapore.