OK, so I started by playing around with Gemini proper. It was fun, or at least fun-ish, but useless as it lacks any sort of storage so using it is like dealing with somebody with Goldfield’s Syndrome a la “50 First Dates”. Every day is a new day.
I was then offered a chance to try Gemini 1.5 for a month for free (and continue for $19.99/month afterwards), with 2 TB of Google Cloud storage included and – according to the offer – 1.5 could help with program development. So I said fine, why not – I’ve been involved with large scale computing from beowulf days, and AI for decades, I was curious as to what it could do – and started it up and began to ask it about its abilities in coding support, what languages it could use, etc.
It literally lied, time and again, about its own abilities. I can’t think of any other word that fits. For example, 1.5 STILL has no access to offline storage for even the paltry, miniscule file sizes needed to store plaintext conversational threads – we’re talking KILOBYTES of data, not even megabytes, at least not quickly, and that’s not compressed. It could – when pressed and corrected a few times – generate functional programs in Octave and C, where C is the language I was really interested in although Octave (free Matlab) was potentially useful to me professionally as well. However, I had to correct its code so frequently that I might as well have written it myself.
Our conversation then led around to a long term project IN the realm of AI – a unique application of neural networks – that I haven’t had time to work on for a decade or so, and I decided to see if G1.5 could facilitate it. Initial discussion went fabulously – it generated on its own almost precisely what my plan was for the application given a fairly general description of the core ideas (which was actually impressive). It then CLAIMED that it could literally generate the code to implement it for GCC using Gnu build tools and open source libraries like the Gnu Scientific Library where possible in a form suitable for me to be able to build it and execute it on my personal (linux) workstation(s).
I actually got excited at the prospect, but there is literally NO WAY to build a complex body of code involving at least dozens of subroutines in their own source files etc using screen scraping from an AI that forgets a conversation entirely if you close the active window where that conversation is occurring. As I said, going slightly mad already… So I asked it for alternatives. And again, it lied, lied, lied.
First it asserted that it could email subroutine or main routine code to me. Lie – no, Gemini 1.5 cannot send email in the course of a conversation that requests it, even to the registered gmail address and account “owning” the conversation. Then I asked if it could access a shared bucket on the Google Cloud if I set one up as I had presumably 2 TB to play with for the rest of the trial period. It enthusiastically agreed that it could! LIE LIE LIE – no, it can’t. I went through the considerable PITA of setting up a bucket, setting up IAMs, configuring rclone so I could mount it on my workstation at home as a presumably shared workspace – I even gave gemini 1.5 the json key file it claimed that it needed for access (no real risk to me as the bucket has no meaningful data in it so far). At the end of several days of work to figure all this out (with the help of gemini itself, but we had to go step by step because one error screwed the pooch time and again) I had it all working perfectly – it is mounted now and I can read and write to it like a linux filesystem.
Gemini, however, cannot. It can’t access it, read files there, list the contents of the bucket etc. Nor can it explain why it can’t. Indeed, it kept claiming that it COULD, lying each time and then failing the simplest tests (like reading back to me the contents of a file it claimed it could read). Ultimately, it became clear that it Gemini 1.5 Pro is completely, entirely useless as a co-developer of code.
At this point I’m really hesitant to proceed and even try things like Code Assist as a tool that appears not to be integrated in any way useful to me. However, I’m willing to be talked out of it if anybody out there can explain what is going wrong or how Code Assist can do what Pro cannot do, and e.g. send email or work on a shared bucket in Google Cloud.
rgbatduke