New York City’s new AI provides illegal advice to businesses when asked for information. Image source: Terabass. via Wikimedia Commons
A bot, named MyCity, was launched in late 2023, and has been found to be providing business owners with information that breaks the law.
MyCity is powered by Microsoft’s Azure AI. The chatbot was promoted as providing “actionable and trusted information”.
Some of the financial advice that MyCity has provided has been to “take a cut of worker’s tips”, “making stores cashless”, and that “there are no specific regulations or requirements that mandate informing staff about schedule changes in New York City”.
This advice ignores legal guidelines concerning cashless stores and worker rights. The chatbot was also found to be unaware that New York’s minimum wage had risen from $15 USD per hour to $16 USD per hour.
MyCity was also found to be informing landlords that they could lock tenants out, increase rent without restriction, and refuse to accept renters based on source of income, such as rental assistance, advice which contradicts New York housing policy.
Andrew Rigie, director of the NYC Hospitality Alliance, has noted that many business owners are confused by the chatbot’s responses.
“I commend the city for trying to use AI to help businesses, but it needs to work,” he said.
“If when I ask a question and then I have to go back to my lawyers to know whether or not the answer is correct, it defeats the purpose.”
The phenomenon of AI chatbots generating and providing false information is recognised as a known issue.
Despite the illegality of information MyCity has provided, current mayor of New York City, Eric Adams, remains supportive of the chatbot.
“It’s wrong in some areas, and we’ve got to fix it. Any time you use technology, you need to put it into the real environment to iron out the kinks,” said Adams.
Adams is a known technology aficionado — openly supporting cryptocurrency, robotic police ‘digidogs’, and AI weapon detectors in subway stations.
Adams is also known for a failed attempt at implementing a “fully autonomous outdoor security robot” program with the K5, created by the company Knightscope. Adams posed with the K5 unit on announcement of the program.
NYC Mayor Eric Adams (D) attempts to make a heart sign with the NYPD’s new “K5 Autonomous Security Robot,” which has no hands or arms to complete the gesture but will soon patrol the Times Square subway station. pic.twitter.com/LVY6SUqzqD
— The Recount (@therecount) September 22, 2023
The K5 was retired to a storage facility after two months. K5 units have been controversial in the past, with a reported incident of a unit having run over a child.
Currently, MyCity’s chatbot website has been updated with disclaimers, informing users that responses may be “inaccurate or incomplete”, and stating that responses were not to be used as “legal or professional advice”.
