Q&A on the topic of AI

There was an important and interesting discussion of AI models a few months back, when deepseek just came out.
And it looks like AI is quickly becoming more and more impactful on all our lives.
So I've decided to take some of my answers in that discussion, and create a more comprehensive Q&A, which I hope people will find interesting.

As I see it, the AI will revolutionize the world, as many other inventions did in the past:
The electricity, the car, the airplane, the computer, the printing press, etc.
But while other inventions initially only affected one or a few fields, it took decades or even centuries until their impact became global on many aspects of our lives.
With AI the progress is much faster, and all of us are either already feeling or will feel within a few years how it affects our lives, jobs, school, etc.

Questions & Answers:

Q: What is AI?
A: AI is Artificial Intelligence.
A computer run software, able to perform human tasks that require thinking, logic, creativity, etc. as well as or better than humans.

Q: How is AI built? What are Neural Nets / Neural Networks?
A: The basis of AI is a concept called "Neural Networks", which is taken from the way our brains work.
We can "teach" it by creating connections between the nodes.
For example, show a child enough pictures of a cat and say the word "cat" and they will start to associate the word with the image (a nueral link is created in the brain)
Similarly, if you show a neural network enough pictures of a cat and let it know it's called "cat", it will eventually will be able to "predict with a high probability" by looking at a picture, if there is a "cat" on it (a neural link will be created in the neural network).
The same principle can be used to train (teach) models not just on images, but also on audio, video, text, etc.
For more information see https://medium.com/@jereminuerofficial/neural-networks-for-dummies-bc1ed3f69027

Q: What is Machine Learning (ML)? What is Data Science (DS)?
A: Machines learning is the profession training various "automated computer models" or "machine learning models". We encompass these models by the general term of AI.
There are many many ways of building machine learning models.
The most common and well known is Neural Networks (explained above).
Data Science is the study subject of training machine learning models (among other things).
People who build/train ML models are usually called "Data Scientist" or "ML model engineer".

Q: How AI models create images/videos/songs/texts? What are GAN (Generative Adversarial Network)?
A: I explained above how Neural Networks are able to learn to "understand" visual, audio or othe input.
Generative Adversarial Network (GAN) is what enables Neural Networks to create output.
We basically pit 2 Neural Nets against each other.
For example, we want to create a Neural Net that can draw Picasso-like pictures.
We take a blank model that can draw, and tell it to draw pictures of random pixels.
We then take one model we trained on existing Picasso pictures, and can tell us a percentage (0% - 100%) how sure it is a picture it's seeing in Picasso.
Each time the first model draws a picture, the second model analyzes it, and gives it a percentage score of how close it is to Picasso. And gives it as feedback to the first model.
Each time the score becomes higher, the first model strengthens the neural link that was used for that image generation.
Each such iteration can take a split second. And as more and more iterations pass, the "correct" neural links are strengthened, and the first model is able to draw pictures that get higher scores from the second model (look more like Picasso).
After millions, or even billions of iterations, the first model will pass a benchmark defined by the ML engineer (for example: above 80% accuracy), which was determined to be "enough" for Picasso-like pictures.
If we run the first model until it reaches 100% accuracy - that would be bad. It's called "overfitting". It means every picture it will create will be a copy of an existing Picasso picture.
And that was not out goal.

Q: What are LLMs (Large Language Models)? Where do they come from?
A: LLMs (Large Language Models) were first created 2 years ago, and used to power the first iteration of ChatGPT.
While ML (Machine Learning) models have been with us for 10+ years. MMLs are relatively new.
The innovation with LLMs is - they showed us a product that's bigger than the sum of it's parts:
You start by taking an ML model that predicts your next word.
Think of writing a message on a cell phone, and it predicts your next word.
But it turns out that if you train it on a data set that is large enough, instead of being a system that is really really good at predicting your next word, but doesn't actually knows how to do anything beyond predicting your next word.
Instead you get a system that can actually write a book like it was Isaac Asimov.
Or discuss any topic you want with you.
Or write a computer program.
Or do the countless different things we use ChatGPT for today.
It's a living proof of the Infinite monkey theorem: a monkey hitting random keys for an infinite amount of time will eventually type out the complete works of William Shakespeare.
LLMs are what powers all of the LM models we call AI (Grok, Perplexity, Dall-E, ChatGPT, Deepseek, Llama, etc.)

Q: How are LLMs different from previous LM models?
A: The main difference of LLMs from previous ML models, is the fact that they are large.
Which is an understatement.
They are HUGE.
And each time they create a new model, they make it more and more HUGE. Like 10x 100x 1000x bigger every iteration.
Which caused several changes in the technological market:

1. Small companies can no longer compete and innovate in regards to AI.
   You cannot be the next Google or the next Meta in AI space.
   Because you need the resources of a behemoth like Google, Meta or Microsoft to create and train a modern LLM.
   Even Apple doesn't have it's own LLM yet. That's how difficult it has become to create/train one.

2. To train a single LLM model, you now need HUGE (HUMONGOUS) computing power.
   Literally thousands or tens of thousands powerful processors.
   And as it's more efficient to do on GPUs (Graphics Cards).
   (For more info see CPU vs. GPU for machine learning)
   Which is also the reason for the rise in the Nvidia stock in the last 2 years: Huge amounts of GPUs are needed to train the LLMs, and the bigger and more complex the LLMs, the bigger the number of GPUs needed for their training.
   And in addition to that, it takes HUGE amounts of electricity.
   To the point that the companies working on new LLM models, are now building their own nuclear power plants, to support their electricity needs, for training AI models.
   Which is why it costs Billions (with a B) of $$$ to develop new LLM models, and only huge & rich corporations can afford to do it.

Q: What is Deepseek? Why are they popular? How are they different from OpenAI?
A: Deepseek is a small start-up which (supposedly) was able to develop their own LLM models.
What was written above, one would presume that this would be impossible, as it would cost them Billions of $$$, which small start-ups don't have.
But they were able to train their model on a "minuscule" budget of only $6,000,000.
It seems they were able to do it, but using a lot of optimization, cutting corners (their model is slightly less accurate than the best ChatGPT model, but is infinitely cheaper) and by using existing models (ChatGPT, Grok, Llama) to participate in training the model.

Q: What is an "Open Source LLM"? How is it similar/different from an "Open Source Software"?
A: Open source software means getting access to the source code of the software.
For example, if software was a table.
Being "Open Source" means you get a full list of parts you would need to build it, and complete step-by-step instructions on how to build it yourself.
If you take the same components, and follow the instructions exactly - you end up with an identical table.
On the other hand, ML models are "black boxes" by definition.
Which means we can train one, but we don't really know or understand how it works inside.
We don't know why it makes one choice and not another - it's all based on the way the millions of neural nodes got linked inside it's brain.
It's like trying to predict what a person is thinking by looking at a CAT scan of their brain.
So "Open Source LLM" is like getting a smartphone with usage instructions.
You can take pictures with it, you can make calls, you can connect it to earphones.
But you can't actually build one yourself based on the instructions you received.
Providing "Open Source LLM" basically allows you to run it on your computer for free (instead of running it on the company's computer, and paying them for it).
But it doesn't provide any information on how it works, or how you can build one yourself.

Q: What are ChatGPT / Grok / Deepseek/ etc.? How are they built?
A: See answer to "What are LLMs (Large Language Models)?"

Q: What's the difference between AI and AGI?
A: TBD

Personal Opinions:

In this section of the Q&A I wilt post Questions & Answers which are not dry facts (like in the section above), but instead represent my personal opinion on the matter.

Q: Is AI going to replace us, make our jobs obsolete, and leave us poor and homeless?
A: The way I see it, we can look at several options:

1. One extreme is - People will be too afraid of AI and will outlaw it.
   This is an extreme option, and I don't think it will happen.

2. The other extreme is - AI will replace humans in jobs, so large percentage of the population will become unemployed.
   There is one problem with this scenario - the one thing politicians love more than money, is being reelected.
   And as long as we live in a world where the people decide what politicians to vote for - there will be enough populist politicians that will do whatever people want. Whatever gets them elected.
   For example, if too many people become unemployed because they'll be replaced by AI.
   It doesn't need to be 80% or 50% of the people.
   Enough it reaches 10% - people will go to the streets.
   Not just the 10%. But their brothers, children, cousins, uncles. People love righteous causes.
   And it's not like police is going to suppress the demonstrations. The police themselves have mothers, brothers and cousins. They don't want them unemployed. So the police will be part of the demonstrations as well.
   So do you think the army will stop them? Think again. Soldiers, officers and generals also have mothers, brothers, cousins...
   Suppressing popular opinion only works when the people demonstrating are a small minority, or when the people suppressing them are a tight-knit group.
   It doesn't work in democratic societies.

3. So what we're left with is option #3: What's going to happen is something in between.
   Like the fact that we have much more advanced monitoring technologies than described in the book "1984" doesn't mean we leave in a dystopian society.
   Same way the fact that AI will be advanced doesn't mean we'll live in a dystopian AI future.
   Think of a Rickshaw taxi driver. All he needs to do is pull the Rickshaw, and get paid for driving people around.
   We put him in a car. So his life is much more complicated now. He needs to learn how to drive. He needs to pay for insurance, he needs to have checkups on his car every year, he needs to full her with gas.
   His life is much more complicated now.
   But he does much less manual labor. And he can get places much faster, which means he can take more people and make more money every day.
   Same way it will happen with AI.
   Let's say someone makes money by drawing advertisement posters.
   He needs to have mental skills to invent a poster, he needs to know exactly what he wants to draw, and he needs to have the physical skill of being able to draw precise lines, to be able to implement his ideas into reality. (into posters)
   Now comes along AI, and takes away his manual labor.
   He still needs to come up with a poster idea. He still needs to visualize it in his head to know what he wants to draw. But he no longer needs to draw it (or have the skill to draw it). He can use AI for that.
   So now his life is more complex. He needs to learn the skills of using AI, of how to write the correct prompt that will cause AI to generate an image in accordance to his vision. But he no longer needs to have the skill of being able to draw himself.

Q: Will AI replace IT workers in the near future? Because they are needed to write codes, but AI may do all of that for them.
A: AI can write code.
But AI doesn't want to write code.
AI doesn't want anything actually. It's incapable of wanting.
It needs a human to tell it what to do.
And the human telling it what to do, needs to understand himself what needs to be done before asking.

It reminds me of a Sci-Fi short story "Ask a Foolish Question" by Robert Sheckley
About a super computer built by an advanced race, that could answer any question in the universe.
But no one asking it questions could get it to answer them the right answers. The computer knew the answers to their questions, but couldn't answer them because they weren't asking the right questions.
In order to be able to ask the right question you need to already know 90% of the answer.

Same with AI.
In order to be able to explain to AI what software you want it to build for you, you need to first be able to build the software yourself.

Feel free to ask additional questions, and I will add them to the Q&A.