In 1772 the Hindu realm Blambangan in the extreme easterly point of Java came under the rule of the VOC. Initially Ulumpangan became the chief settlement of the company, but in 1790 the company moved her headquarter to Banjuwangi on the Bali Straits. Here a military post was built near the mouth of the river. Another important building in the city was the Bandit’s place for people because Banjuwangi was a penal colony. In 1801, Captain J. Schmaltz of the Engineer Corps designed a large fort to replace the small blockhouse, but is was never built. In 1811 Banjoewangi became the terminus of the Great Post Road of governor general Daendels. The Blockhouse Utrecht was reinforced. During the British period (1811-1816) it was a small fort with 150 soldiers. An attack of pirates was repulsed.
After the return of the Dutch in 1816 they laid a garrison in the fort. In the 1840s, the inside of the fort included one house for the officers, one barrack for European and one for native soldiers, two warehouses, one powder-house, a guardhouse and a military prison. There was a garrison of 1 First lieutenant, a Second lieutenant, 44 native infantry, six European and 14 native artillery soldiers. In the barracks also lived 31 women and 21 children. The fort had nine cannon: three twelve pounders and six six-pounders. Shortly afterwards the fort became a prison and a school.
The gate of fort Utrecht was dismantled between 1927 and 1929. The other buildings have also largely been demolished. The only remains of the ruins are walls, which are now part of houses of the local people.