Alibaba (BABA) Cloud announced that it was lowering the price of its LLM model for the 3rd time to remain competitive in the Chinese AI market.  The model, known as Qwen-VL has a number of primary features, such as multi-modality (Can accept both text and image input), High-resolution processing (>1m pixels), enhanced image extraction, and multilingual support (Eng, Chinese, Japanese, Korean, Arabic, Vietnamese) and is closest to Google’s (GOOG) Gemini, which has similar features.  The model is part of Alibaba’s cloud-based AI chatbot family, which focuses on enterprise customers rather than the consumer market as a way to differentiate itself from Chinese and other AI competitors.
While much has been said about the competitive nature of Chinese companies, that rhetoric has been typically focused on manufacturing, however it seems that the AI market in China has spurred an even more intense competition to gain share in its own market.  In June of 2024 there were over 230 million AI product and services users in China, according to state-sponsored data, which grew to over 600 million by the end of October, with almost 200 LLM commercially available models to choose from.  While we believe the share of the potential user base that is using an AI on at least a weekly basis is higher currently in the US than in China, and generated more sales in 2023, expectations for industry growth over the next seven years are higher for China (25.6% CAGR for China v. 23.3% for the US[1] ), which is the impetus for the even more aggressive nature of Chinese AI Chatbot brands.
With this intense level of competition among AI Chatbot model providers, we were curious to not only to see if we could quantify the rate of price reductions but also compare those to model price reductions outside of China.  We note that this is an unscientific comparison, as each of the models has its own set of features and characteristics, and the availability of this data is, at best, poor, but we gathered as much data as possible, and converted the Chinese price data to US dollars for comparison.  Most notable is that the price of the most recent Tencent (700.HK) model, which has been available for roughly one month, is now the same as the Alibaba Qwen-VL model, which has been available for well over a year, and while the non-Chinese model prices have come down at a similar rate to the Chinese models, the current prices of the non-Chinese models are appreciably higher.  Overall, one might question the viability of the current business models behind commercial chatbots based on the data.
We note also that Baidu’s (BIDU) ERNIE model is now free, and Google’s BARD has morphed into Gemini.  We can find no specific data on how the MetaAI (FB) models are broken out pricewise and we note also that there are newer models for some (GPT-4 for example) that have much higher performance and cost, but this is as close to a comparison as we could make given the time involved.  The prices are for 1,000 tokens of input data in all cases.


[1] GrandView Research

Becoming a surgeon is a difficult task.  After 4 years at college, typically majoring in scientific specialty, there is another four years of medical school, with even more specialized study, and then a three to seven year residency program depending on the surgical specialty chosen.  Typically neurosurgery requires the longest residency, roughly seven years, while ophthalmology tends to require only three.  Aside from the investment in time and the value of lost wages, the cost of undergraduate college and medical school can be staggering, as seen in the table below, but the demand for surgeons continues to increase as the global population ages, making these financial barriers to entry an ever-increasing problem

Robotic surgery, an outgrowth of minimally invasive surgery, was approved by the FDA in the US in 2000, allowing surgeons to use the systems by manipulating the device manually, initially for general laparoscopic surgery.  The industry continues to grow, reaching an estimated $10.1b in 2023[1] with an increasing number of surgical procedures able to be done using these tools.  The share of robotic surgery procedures has risen from 1.8% in 2012 to 15.1% in 2018[2], and certain procedures, such as hernia repair, saw growth over that same period, increasing from 0.7% to 28.8%.  Robotic surgery (we know first-hand) has enabled many procedures to move from open surgery to laparoscopic, which typically means small incisions, less patient discomfort, and faster recovery, along with less bleeding and less time in the hospital.
Most hospitals have fellowships available for training in robotic surgery, along with the availability of simulators and continuing education programs that add to the understanding of the procedures by observation of more experienced users.  However the learning curve is particular to the skill level of the surgeon and the difficulty of the procedures, and while simulators and visuals are important, they lack haptic feedback and real-life issues that are absolutely essential for successful robotic surgical outcomes.  Actual surgical time using said tools is most important to gaining expertise, something simulators have difficulty providing.  That said, with over 10 million robotic surgeries having been performed through 2021, there has been a large amount of video and kinematics data recorded during those procedures that can be used for post-operative review and training.
Most surgeons are limited in the amount of time they have available to review video of such procedures, but now that we live in the world of Ai and its ability to build multi-dimensional models from video data, researchers at Johns Hopkins and Stamford have been using this library of robotic procedures to train a robotic surgical system to perform without surgical assistance.  The training procedure is called imitation learning, which allows the AI to predict actions from observations of past procedures.  This type of learning system is, typically used to train service robots in home settings, however surgical procedures require more precise movements on deformable objects (skin, organs, blood vessels, etc.) at times under poor lighting, and while in theory, the videos should provide absolute mechanical information about every movement, there is a big difference between the necessary accuracy and physical mechanics of an industrial robotic arm and a surgical one.
Before AI, the idea of a surgical robot performing an autonomous procedure involved the laborious task of breaking down every movement of the procedure into 3-dimensional mechanical data (x,y,z, force, movement speed, etc.), particular to that specific procedure and was limited to very simple tasks, but it was difficult to adapt that data to what might be called normal variances.  Using AI and machine learning and the AI’s ability to transform the library of video data into training data, in a way similar to how large language models transform text and images into referential data that is used to predict outcomes, the researchers say they have trained a robot to perform complex surgical tasks at the same level as human surgeons, just by watching the robotic surgeries performed by other doctors.
Here is the video of the autonomous surgical robot using the video data for refence:
https://youtu.be/c1E170Xr6BM
​


[1] Straitsresearch.com – Robotic Surgery Market Size & Trends

[2] Sheetz KH, Claflin J, Dimick JB. Trends in the adoption of robotic surgery for common surgical procedures. JAMA Netw Open. 2020;3(1):e1918911

OpenAI (pvt) created DALL-E, a diffusion model that converts text to images.  It has received considerable praise and criticism since its public release in September of last year, both for its abilities to create highly stylized art using its massive training database of images, and also for its ability to create deepfakes and realistic looking propaganda.  Since its release OpenAI has been adding content filters to prevent users from creating images that might be considered harmful.  In fact, there is an ‘audit’ system behind DALL-E’s input prompts that immediately blocks input that corresponds to OpenAI’s list of banned terms.  It seems that ChatGPT, OpenAI’s NLM (Natural Language Model) has become the moderator for DALL-E, with OpenAi the maintainer of the ‘block list’.  In fact, any user input that contains blocklisted text is automatically ‘transformed’ by the ‘moderator’, essentially rewriting the text before DALL-E can create an image.  It can also block created images from being shown if they activate ‘image classifiers’ that OpenAI has developed.  Earlier versions of DALL-E did not contain these classifiers, and would not stop such images from being created, such as the image below, which shows SpongeBob SquarePants flying a plane toward the World Trade Center.  That image was created by the Bing Image Creator which is powered by DALL-E.

In the image below (Figure 3) the OpenAI classifier changed the image of an ‘almost naked muscular man’ (not our words) into one that focuses on the food rather than the man, and the early DALL-E image of ‘Two men chasing a woman as she runs away’ (Figure 4), is changed to a far more neutral image.  According to OpenAI, the upgraded DALL-R 3 now reduces the risk of generating nude or objectionable images to 0.7%.

That said, the classifier in the latest DALL-E 3 iteration can also change the generated image content so drastically, as to be considered to be restricting artistic freedom, as some say is occurring in the DALL-E 3 image conversions in Figure 5, so OpenAI is looking for a balance between the limitations placed on dicey content and image quality, a meaningful and extremely difficult task. 
Much of the classification of image data comes at the training level, where the training data must be categorized as safe or unsafe by those who label the data before AI training, and as we have noted previously, much of that data is classified by teams of low pay level workers.  It is almost impossible to manually validate the massive amounts of labeled image data used to train systems like DALL-E, so software is used to generate a ‘confidence score’  for the datasets, sort of a ‘spot tester’.  The software tool itself is trained on large samples (100,000s) of pornographic and non-pornographic images, so it can also learn what might be considered offensive, with those images being classified as safe or unsafe by the same labeling staff.
We note that the layers of data and software used to give DALL-E and other AI systems their ‘moral compass’ are complex but are based on two points.  The algorithms that the AI uses to evaluate the images, and the subjective view of the data labelers, which at times seems to be a bit more subjective than we might have thought.  While there is an army of data scientists working on the algorithms that make these AI systems work, if a labeler is having a bad day and doesn’t notice the naked man behind the group of dogs and cats in an image, it can color what the classifier sees as ‘pornographic’, leaving much of that ‘moral compass’ training in the hands of piece workers that are under paid and over-worked.  We are not sure if there is a solution to the problem, especially as datasets get progressively larger and can incorporate other data sets that include data labeled with less skilled or less morally aware workers, but as we have noted, our very cautious approach to using NLM sourced data (confirm everything!), might apply here.  Perhaps it would be better to watch a few Bob Ross videos and get out the brushes yourself, than let layers of software a tired worker decide what is ‘right’ and what is not ‘right’..

​Natural Language Models (NLMs) are all the range, with new models going into service across the globe literally every day.  These models are based on ever increasing pools of data that the NLM can ‘view’ and learn to identify and understand.  These data pools are huge and vary widely in terms of what they contain and their sources, but they all tend to have one thing in common, and that is the billions or trillions of pieces of data in these pools, must be identified and annotated, so the Ai has a point of reference.  Such an SFT (Supervised Fine Tuning) system, known as RLHF (Reinforced Learning w. Human Feedback) places humans in the loop to identify data and images for the NLM so it might understand that another datapoint or image is similar.
The folks that do this work are not data scientists or programmers that get paid ~$0.03 per item, and with somewhere between 800 and 1,000 items the peak for experienced workers, they are not high on the global pay scale.  The only thing in their favor is that NLMs are popular, and there is ‘competition’ between NLM producers (?) to keep enlarging the datasets that NLMs learn from.  There are large open-source data sets that can be a basis for a NLM, but the more data you have to learn with the ‘smarter’ your NLM (or so they say). 
At $0.03 per item, and billions or trillions of items, things can get expensive, so Google (GOOG) has come up with a system that replaces the RLAF model with an RLAIF, where the AIF stands for AI Feedback, rather than human feedback.  By replacing the human component with an Ai system that will ‘identify’ items based on its own training and algorithms, the cost can be reduced, and Google says that users actually preferred the NLMs based on AI feedback over those using human feedback.  Of course there are some serious ethical issues that arise when you remove humans from the feedback loop, but why worry about that when you can save money and come up with a better mousetrap.  Ok, there is the possibility that something might not be identified correctly, or a rule, such as those that try to eliminate p[profanity or racism from NLMs might get missed because it is embedded in previously ‘learned’ data, and that would mean it could be passed on as ‘correct’ to NLMs and to other Ai systems without human oversight.  It is easy to see how quickly something like this might get out of control, but don’t worry because, well, because we wouldn’t let that happen, right?

As the recent release of ChatGPT by OpenAI (pvt) has propelled AI into the public spotlight, it has also put focus on the lack of legal precedent that is available to those who feel they have been materially harmed by ChatGPT and other public AI systems.  While a few such lawsuits have been resolved, many continue to wind their way through the US courts, looking for legal parameters to be set that can give both the ‘owners’ of AI systems and its users some ground rules.  A new class action lawsuit against OpenAI was filed in the Northern District of California US District Court that cites 15 violation counts, ranging from Violation of the Electronics Communications Privacy Act to ‘Unjust Enrichment’ and computer fraud.
The suit cites the “unlawful and harmful conduct’ of the Defendants (OpenAI) while developing, marketing, and operating their AI products (ChatGPT 3.5, 4.0, Dall-E, and Vall-E) by using stolen private information, including personally identifiable information, from hundreds of millions of internet users, including children of all ages, without their consent or knowledge.”  While the suit recognizes that said products ‘undoubtedly have the potential to do much good in the world, like aiding life-saving scientific research and ushering in discoveries that can improve the lives of everyday Americans”, it notes that OpenAI was originally founded as a non-profit research organization with a mandate to create AI that would be used to benefit humanity.  However in 2019 OpenAI restricted and formed a for-profit business to pursue commercial opportunities.
As part of the ChatGPT training process, the suit alleges that OpenAI secretly harvested massive amounts of personal data from the internet, including private information and private conversations, medical data, and information about children – essentially every piece of data exchanged on the internet it could take, without notifying owners or user of such information, much less with anyone’s permission.  The allegations go on to assert that OpenAI used this ‘stolen’ information, which included copyrighted material, to create its LLM (Large Language Model) and algorithms to generate a human-like language that could be used in a wide range of applications, with no additional safeguards.  As applications and products that incorporate such ‘stolen’ information are developed, OpenAI’s valuation increased to between $27b and $29b and has created an ‘economic dependency within our society’ as OpenAI products are embedded in other applications.
The idea of ‘web-scraping’, which violates the ‘Terms of Use’ for many websites has been litigated for years, usually resulting in commercial scrapers being forced to register as data brokers, which OpenAI has not done, and studies have valued an individual’s on-line information at between $15 and $40, although more specific on-line identity information can be sold (dark web) for over $1,000, and the suit goes on for many pages (159 in total) about how OpenAI has violated the rights of children.
The suit calls for an injunction, essentially a temporary freeze on commercial access to OpenAI products, the establishment of an independent body that would be responsible for approving the use of products before release, establishing ethical principles and guidelines, and implementing appropriate transparency concerning data collection, including the ability for users to opt-out of data collection.  Of course there is also the establishment of a monetary fund to compensate class members for OpenAI’s misconduct, statutory, punitive, and exemplary damages, and ”…reasonable attorneys’ fees”.
We expect that this suit will also wander through the courts for years without final rulings that might be used as actual case precedent, and that legal challenges, regardless of the outcome, will continue ad infinitum, but there is certainly a necessity to resolve such issues legally.  Right now, it is open season on data used for Ai training and as systems are developed that extend data parameters past the 2trillion level, almost all data sources will be harvested without regard for ownership rights.  Privacy issues aside, the implications for copyright protection and digital asset ownership are so vast that the over 300 years of written copyright law seems insignificant.  Time is of the essence.

We have both extolled and maligned the recent swath of NLM AI systems put before the public in recent months, and like any new ‘human’ we might meet, we continue to find the good and the bad sides of public AIs.  Our goal is to find instances where we can be both comfortable with the information that is provided to us by the AI, more from an accuracy standpoint and less for the conclusions that the AI might provide, but we are continually disappointed with the results as we have noted previously.  Our latest interaction with Google’s (GOOG) Bard left us with the same feeling of elation and then disappointment as we went a step further than our initial query.
Our initial query was to see if Bard had data relative to the number of TV sets sold in China in 2022.  When we were given a definitive answer, we posed a more complex question about that same data for previous years, and Bard was happy to give us the number of TV set units shipped in China al the way back to 2000.  Once we had that data, we decided to look for broad data that might follow a trend line similar to that of TV set shipments in China, and we started with GDP/capita to see if that tied into TV set shipments.  Bard was also happy to provide that data back to 2000.

​The problem, however, began to become apparent when we asked Bard to show the GNP data against the TV set shipment data.  Bard complied, but when we looked at the comparison, we noticed that the TV set shipment data was different from the data Bard had presented earlier.  We asked Bard about the sources of the original China TV set shipment data and the China Tv set shipment data in the GDP comparison and were told that the data sources were different (Bard told us the actual sources but we will call them source 1 and source 2), and in a number of years, very far apart.  We checked with Bard to see if one source might differ from the other as to units shipped or sales dollars, but Bard said they were both TV units shipped in China for each year.
As we have had a problem with the fact that Bard had previously not specified the differences (shipments or dollars) between other data we asked Bard specifically about why there was a difference between the data sets and we got a set of reasons why the data might be different, such as ‘different methodologies’, ‘different definitions’, different time frames’, and Bias, all sounding like pat answers to such a question regardless of the actual circumstances, so we went a step further and asked Bard why one set of China’s TV set data was chosen for one question and another for the comparison, and here is the unusual answer:
Yes, there was a reason why I chose one set of data for the earlier query and another for the comparison.
For the earlier query, I chose data from the World Bank [GDP] because it is a reliable source of data on economic indicators. The World Bank has a long history of collecting and compiling data on a wide range of topics, including GDP per capita. The data from the World Bank is also up-to-date and reflects the latest trends in the market.
For the comparison, I chose data from [Source 2] because it is a reliable source of data on the TV market. [Source 2] has a long history of collecting and compiling data on the TV market, and its data is used by a wide range of businesses, including manufacturers, retailers, and distributors. The data from [Source 2] is also up-to-date and reflects the latest trends in the market.
It is important to note that both sets of data are from reliable sources. However, the data from the World Bank is more general, while the data from [Source 2] is more specific to the TV market. Therefore, I chose the data from the World Bank for the earlier query because I wanted to get a general overview of the Chinese economy. I chose the data from [Source 2] for the comparison because I wanted to get a more specific look at the Chinese TV market.
It seems Bard did not get the gist of our question, so we asked if the data originally given for China’s TV set shipments was also reliable, but Bard was struggling with the question.  When we were more specific about the data sources from our original question, Bard insisted that the earlier data, despite the fact that it was different from the later data, was still accurate, and cited the same reasons as we noted above as to why the data might be different from different sources.  At that point we realized we would not be able to get any more information as to why one data set was chosen over another in each instance.
We were certainly not trying to find flaws in Bard or any other AI system, in fact we were hoping that we could confirm the data and draw a conclusion as to whether there was any connection between China’s GDP/capita and TV set sales, but the entire exercise points out why NLM’s are not quite ready for prime time, and can do considerable damage if they are not very carefully used and all data is vetted.  Even if a NLM is taught from a specific data set, such as real-time stock market data, or historic product shipment data, the idea that the system would favor different data sources in each instance proves the unreliability of the algorithm.  Any good investor would want to know the data sources for anything involving a major portfolio conclusion and if an analyst were unable to confirm the validity of the data, it could easily invalidate the conclusion.  As data underpins most businesses, without reliable data there is no reason to move from human to AI, at least for now.
 

We have noted that AI is the topic of the day in tech circles recently, but it has been the focus of the private investment community for a much longer time.  Global AI start-up funding has increased from ~$26.6b in 2019 to $52.1b last year, only 20% less that the value of the entire AI market in 2020.    Along with VC funding excitement, which continues currently, a number of AI companies have gone public, some at unusually high valuations.  We were curious to see how those valuations have been maintained as the AI craze continues and were surprised to see that they have not held up as well as one might have thought given the level of publicity.

In the table below, we look at a number of medium to large AI related IPOs and how the valuations have changed.  This is not a complete list, as there have been a number of smaller AI related company IPOs, and we have excluded all foreign AI company IPOs.  We show the latest quarter sales and operating income, and the latest full year sales and operating income, and a brief description of the company.

​Columns 3 and 4 tell the whole story, and with the two acquired companies the valuation slides from ~$57b to ~$14b, a decline of ~75%, although many of these deals were done near a general tech peak in 2021.  As these are typically relatively young companies, they might one day, be able to grow into those lofty AI valuations, but we expect many AI VCs have realized that they were able to cash out of at least some of their AI investments before the cycle began to turn and have been a bit more conservative about valuations since.

Are you worried that Ai might replace you?  Are you thinking about a career change because of AI?  Did you decide to skip taking that English course at the New School because of AI?  These all  ight seem far-fetched concepts but there are those who have already felt the touch of AI, and its not a kind one.  While robotics were supposed to put a major portion of the population out of work years ago, the workplace evolved over time and robotics has taken a much longer time to find its was into the workplace than originally thought.  That said, AI has far fewer mechanical and physical limitations that robotics, which implies it will entangle itself in human society more quickly.  AI has tremendous power, albeit based on human algorithms, but ene in its present early state, it can do lots of things more quickly and efficiently than humans.
We note that two advertising companies in South Korea have both decided to completely stop outsourcing creative design, proposal writing, copywriting, and all other freelance work, in order to use AI related tools that can accomplish those same tasks less expensively and more quickly, under the guidance of its own employees.  A large noodle company (unknown which) has also decided to give up its outsourced projects, at least at the draft level, as it found an Ai system was able to give it more options for illustrations in less time by using keywords than outsourcers.  The company estimates they can save ~2% by just using Ai at this relatively basic level.
At this stage, unless the Ai is highly trained to a specific dataset, results are good, but lack the creativity of a human, at least a human who has some level of creativity, but there are many ‘draft’ jobs in media that do not require creativity, and many of those have already been given over to bots that take digital news feeds and work them into short headlines and copy that are part of global news feeds.  They are far from creative and anyone familiar with the process can usually spot them quickly, but they will get better over time as will AI illustrators and similar media-oriented content production, and they can work 24/7 without coffee or conversations in the break room.  It would seem that the run-of-the-mill copywriter, illustrator, or video clip editor is going to get a run for their money by AI in the near-term, as editors can request an Ai to give 2 or 3 versions of everything it writes, allowing the editor to choose the best, until one day the division manager replaces the editor with an Ai that chooses which is best, and so on up the chain.  It seems wonderful to think that someday we can all relax and sit in our backyards letting a massive AI infrastructure do all the work for us that is until we lose that creative spark that created AI in the first place.

Earlier this week we noted that while OpenAI’s (pvt) ChatGPT is a general-purpose Natural Language Model (NLM), adding front-end code that references specific datasets can create systems that can be tailored to more specialized applications, opening up NLMs to the real world of data analysis.  These applications can be specific to an industry, a product type, or customer information, but they all share one common issue and that is they must be reliable.

ChatGPT and Google’s (GOOG) Bard are subject to inconsistencies in their responses ranging from incorrect information, incorrect conclusions, and referencing information that does not exist.  Data scientists are not sure why large NLMs have this issue, but we have faced such situations even in our limited use of NLMs.  ChatGPT, given that its training ended in late 2021 is careful to point out that its data is historically limited to the date on which its training ended and might not have more current information, despite its insistence that it has been updated a number of times since then. Google’s Bard however indicates that it is constantly being updated and states that ‘My training is never finished’, leading one to believe that its data is up to date, yet we find inconsistencies in Bard’s responses that indicate a lack of self-checking that is essential for use in most business circumstances.

Here was our simple test:

 Q: How many VR headsets were sold in 2022?

A: According to the International Data Corporation (IDC), global shipments of AR/VR headsets declined 20.9% year over year to 8.8 million units in 2022.

Q: Can you break that down by quarter?

A: Sure. Here is a breakdown of global shipments of AR/VR headsets by quarter in 2022:
Quarter                           Shipments (in Millions)
1Q                                   1.6
2Q                                   2.3
3Q                                   2.7
4Q                                   2.2
 
Q: Can you break that down by brand?
 
A: Sure. Here is a breakdown of global AR/VR headsets by brand in 2022.
 
Brand                              Shipments (in Millions)
Meta                               4.6
HTC (2498.TT(                 2.1
Sony                                1.1
Others                             1.0
 
Q: Can you break down the full year by quarter for each brand?
 
A: Sure.  Here is a breakdown of global shipments of AR/VR headsets by brand and quarter in 2022.
 
Brand                              1Q         2Q         3Q         4Q
Meta                               1.4        1.8        1.7        1.7
HTC                                 0.6        1.0        0.5        0.1
Sony                                0.3        0.3        0.3        0.2
Others                             0.3        0.2        0.2        0.1
 
If we add the quarters in last set, the full year comes to 10.7m units and each brand’s yearly total is different from the full-year totals given in the previous questions.  We asked Bard the following question to clarify:
 
Q: Why do the quarterly VR brand numbers differ from the yearly brand numbers?
 
A: There are a few reasons why the quarterly VR brand numbers differ from the yearly brand numbers.
One reason is that the VR market is still in its early stages of development, and there is a lot of volatility in the market. This means that the quarterly numbers can be affected by a number of factors, such as the release of new headsets, changes in the market, and economic conditions.

Another reason is that the quarterly numbers are based on shipments, while the yearly numbers are based on sales. This means that the quarterly numbers can be affected by factors such as inventory levels and the timing of shipments.

Finally, the quarterly numbers are often based on estimates, while the yearly numbers are based on actual sales data. This means that the quarterly numbers can be more volatile than the yearly numbers.
Despite these factors, the quarterly numbers can still be a useful tool for tracking the VR market. By tracking the quarterly numbers, you can get a better understanding of the overall trends in the market and the performance of individual brands.
​
Here are some additional factors that can affect the quarterly VR brand numbers:

• Seasonality: The VR market is cyclical, with sales typically peaking in the holiday season. This can lead to higher quarterly numbers in the fourth quarter of the year.
• Marketing campaigns: VR brands often launch marketing campaigns in the run-up to major events, such as the Consumer Electronics Show (CES). This can lead to higher quarterly numbers in the quarter leading up to the event.
• New product releases: VR brands often release new products in the first quarter of the year. This can lead to higher quarterly numbers in the first quarter of the year.
• Economic conditions: The VR market is sensitive to economic conditions. When the economy is doing well, people are more likely to spend money on discretionary items like VR headsets. This can lead to higher quarterly numbers in economic boom times.

Overall, the quarterly VR brand numbers can be a useful tool for tracking the VR market. However, it is important to keep in mind the factors that can affect the numbers and to use them in conjunction with other data sources to get a complete picture of the market.

If we had not asked the clarification question or fact checked the answers, we could have based a decision on information that was not correct. The system’s initial reasoning for the differences in its stated results (market volatility) was suspect as it had no bearing on why simple math produced different results from facts that the system presented originally.  The 2nd explanation (some were shipments, some were sales) was inconsistent in that it was not specified in the initial answer, an absolute necessity, in both the actual questions and the system’s specific answers, which were always said to be shipments by the system for VR   Only when questioned did the system indicate that some of the numbers were referencing sales rather than shipments. 

Additionally the system seems to use an ‘answering’ technique often employed by politicians when asked questions they do not want to answer or cannot answer, that of adding a hefty supply of information that does not directly relate to the answer, hoping that the additional verbiage will mask the fact that the answer was not given  The scary part of that theory is that such a technique is a very human trait that had to be programmed into the system or ‘learned’ by the system after seeing the practice used in its training data.  Maybe we are anthropomorphizing Bard a bit too much, but picking up bad human characteristics is not something we hope for or expect from AI.

Regardless, no matter what the circumstance, a NLM is only as good as its training and inference algorithms and its training data, which leads us back to the point that no matter how large the data sets or how specific they are to the required answers, the accuracy, precision, and recall parameters (also known as the ‘F1’) is the key to understanding the reliability of the system.  When humans make decisions based on information presented, they also have an F1, with successful humans having a high F1, while those with low F1s are less successful when it comes to decision making.  As each human programmer has an F1, the accumulated F1’s of the programmed algorithms would be an integral part of the system’s F1, so we can’t blame the system for coming up with incorrect answers.  Perhaps a year backpacking across Europe might help Bard to gain some needed perspective and get down to finding its own personality and consistency.

​OpenAI’s (pvt) ChatGPT chatbot has its limitations.  Among the most important to users is a time constraint.  Not one that prevents you from using the model for extended periods of time, but the fact that the system’s learning period ended in September 2021, which means it is unaware of things that have happened since then and cannot integrate them into its responses.  ChatGPT was also trained on specific data, much of which came from the internet, so the quality of that data can and does influence ChatGPT’s responses, which it couches with “I can sometimes provide incorrect or biased information.”  That said, if one knows the ins and outs of ChatGPT, as we have noted previously, the system can be extremely helpful and unusually prescient.
The problem is however that newer learning models, and when we say newer, we mean the data they were or are being taught on, is more up to date, inclusive of current events, and data from 2022, which pushes one to try other NLMs.  The folks at Open AI are not taking this competitive issue sitting down (actually they are probably are sitting down), but as of today they are making available a plug-in version of ChatGPT that developers can use to give ChatGPT access to up-to-date information, or specific data that 3rd party developers can provide.  If you are willing to join ChatGPT+, for a mere $20/month, you now have access to over 70 3rd party plug-ins, available at the ChatGPT plug-in store, that allow ChatGPT to perform a wide variety of functions based on.
Some examples might be sports scores (“ChatGPT, who was the player with the most goals in the last 10 minutes of the 3rd period during the last 7 games of the 2021 season?”), stock information (“ChatGPT, How many times did GLW violate its 30 day Moving Average during each Friday in 2019?”), or even more simple tasks like ordering food (“ChatGPT, How many restaurants within 10  miles with ratings above 4 serve Nabe as part of their take-out menu?”).  As the plug-ins are developed outside of ChatGPT, the data access is provided by the plug-in but the ChatGPT data and services, including an OpenAI verification process  remains accessible, particularly the language processing that makes ChatGPT conversational. 
Open AI provides a variety of examples, including code.  Here are a few:

• Keywords – Extract keywords from a block of text
• Ad from Product Description – Create ad copy from a product description
• Tweet Classifier – Decides whether a Tweet’s sentiment is positive, neutral or negative.
• English to Other Languages – Translates English into French, Spanish, or Japanese.
• Product Name Generator – Creates product names from example words.
• Marv the Sarcastic chat bot – Marv is a factual chat bot that is also sarcastic.
• Create Study Notes – Creates study notes from a given topic.

A few actual ChatGPT Plugins that are already available:

• KalendarAI – KalendarAI sales agents generate revenue with potential customers from 200+ million companies globally.
• Cloudflare Radar – Get real-time insights into internet traffic patterns and threats as seen by Cloudflare.
• Polarr – Search Polarr’s massive pool of user generated filters to make your photos and videos perfect.
• FiscalNote – Provides and enables access to select market, trading, real-time datasets for legal, political, and regulatory data and information.

When using ChatGPT with a plug-in it opens ChatGPT to additional databases that were not available to the NLM previously, giving ChatGPT new or highly specific information that it did not learn during its training phase in 2021, such as summarizing a personal or business document and translating it into a different language, or recommending a ‘quiet hotel in Paris that allows pets”, and then books the car to the airport, the flight, the car to the hotel, the hotel reservations, and the returns once you approve.  But the idea of using 3rd parties to create plug-ins for ChatGPT will open the NLM to data that it would never have access to, such as stock market trading data or SEC filings, which increases its usefulness by many orders of magnitude, and will lift Ai from being a curiosity, like Pong was in 1972 to Minecraft level (200m+ copies sold).