How well does AI evaluate astrology programs?
By Hank Friedman
I was asked today to compare two astrology programs for a potential customer. Since I know both programs "like the back of my hand", I was curious how the top AI engine would compare the two, and generated the query.
AI got an "F".
The report was full of bad advice, attributing features belonging to one program to the other, ignoring many crucial details, trusting and regurgitating the creator's PR hype when it shouldn't, not mentioning or taking into account many of each programs capabilities, and otherwise completely misguiding the reader, especially in the final summary.
Anyone following the report's advice would be very disappointed.
Most people don't realize that AI doesn't really think or understand what it is evaluating. It is a very sophisticated information collection tool, but bases its output on what it reads.
So when an AI engine reads misinformation on the web, for example, it doesn't know it's misinformation!
I like AI in general, and have used it successfully for programming snippets, some product evaluations (e.g. ultrasonic devices, toothbrushes, etc.), and such; but in my explorations I've found it to be accurate in its guidance about 60 - 75% of the time, and otherwise way off.
Comparing the extensive features of two astrology programs is simply way beyond the capabilities of AI models.