Non-Technical Introduction to AI Fundamentals
From the lens of Nikolai Kondratiev's theory of economic cycles, AI is poised to propel us into a new wave of prosperity. Similarly, AI is surpassing the 'peak of inflated expectations' in Gartner’s hype cycle and inching closer to widespread adoption.
William Gibson once said, “the future is already here – it's just not evenly distributed.” With AI, the most powerful applications are currently reserved for those with astronomical budgets and access to near-infinite computing power. But, thanks to restless efforts of researchers, developers and entrepreneurs, that is changing very quickly.
ChatGPT broke records in consumer adoption. It achieved one million users in just five days, and 100 million users two months after its launch. Generative AI, which includes ChatGPT, is on the verge of "crossing the chasm" from early adopters to early majority. The key to this transition lies in proving the utility of AI concepts, finding the right pricing models, and ensuring accessibility.
Whether you're in finance, healthcare, retail, or any other sector really, AI impacts or will impact your work in some way. But for those without a tech background, the jargon can often be overwhelming. This artificial intelligence tutorial aims to bridge that gap and offer a comprehensive look at AI fundamentals.
Essential terminology
Artificial intelligence
Artificial Intelligence, often abbreviated as AI, refers to the capability of a machine to imitate human-like behavior. Think of it as teaching machines to "think" and make decisions, somewhat like a human would. AI is the umbrella term that houses various applications and technologies, including machine learning and deep learning.
Machine learning
How do you teach a computer to play checkers? This was actually one of the first working applications of machine learning algorithms, developed all the way back in 1959 by Arthur Samuel.
Machine learning is essentially a subset of AI that allows a computer system to learn from experience. In simpler terms, machine learning is about using data to answer questions. For instance, a machine learning algorithm might use sales data to predict next month's revenue.
Machine learning models work by:
- training a model on a set of data,
- using that model to make predictions or decisions.
Example applications of machine learning:
- Fraud detection in finance
- Recommender systems for e-commerce
- Predictive maintenance in manufacturing
- Natural language processing in customer service
- Image recognition in healthcare
- Autonomous vehicles in transportation
- Chatbots in customer relations
- Speech recognition in voice-operated systems
Deep learning
Deep learning takes machine learning a step further. It employs more complex algorithms and uses an AI concept called deep neural networks to train the model. In a few words, how deep learning works is similar to how the human brain operates, with neurons, layers, and so on.
Essentially, deep learning can make sense of even more complex data sets, making it highly valuable for tasks that require a deeper level of data comprehension.
Example applications of deep learning:
- Image and speech recognition
- Natural language processing
- Fraud detection
- Recommender systems
- Autonomous vehicles
- Medical diagnosis
- Financial modeling
Computer vision
Vision systems in AI are designed to interpret and understand visual data from the world, such as pictures and videos. They can identify objects, read signs, and more.
Applications of vision systems:
- Object detection
- Facial recognition
- Image and video analysis
- Autonomous vehicles
- Medical imaging
- Quality inspection
- Surveillance
Natural language processing
Natural language processing (NLP) is a branch of AI that gives machines the ability to read, understand, and generate human language. This technology powers chatbots, virtual assistants, and other services that interact with people in a natural way.
Applications of NLP:
- Chatbots and virtual assistants
- Sentiment analysis
- Language translation
- Speech recognition
- Text summarization
- Customer service
Different Types of Algorithms
Regression algorithms
These are used for predicting numerical values, such as house prices. Examples include Linear regression, K-NN, and random forest.
Classification algorithms
These are used to classify data into different classes, like diagnosing a disease. Examples are Logistic regression, random forest, and K-NN.
Clustering algorithms
These categorize unlabelled data into similar groups. Examples are K-Means and DBSCAN.
Time-series forecasting algorithms
These predict future values based on past time-series data. Examples include ARIMA and LSTM.
Anomaly detection algorithms
These find outliers or unexpected items in datasets, often used in fraud detection. Examples are Isolation Forest and Local Outlier Factor.
Ranking algorithms
These order results based on certain criteria, such as recommendation engine outputs. Examples are Bipartite ranking methods like Bipartite Rankboost.
Recommendation algorithms
These recommend next items, like songs or videos. They generally use Content-based and collaborative filtering methods.
Data generation algorithms
These generate new data, like images or videos. Examples include LLM and Generative Adversarial Networks (GANs).
Optimization algorithms
These are used to find the best solution to a given problem based on an objective function. Examples are linear programming methods and genetic programming.
Designing AI-Based Products
Data labeling and preparation
Before diving into AI project steps, it's crucial to note that data is at the heart of every AI initiative. In fact, over 80% of the time in AI projects is often spent on data labeling and preparation. So, before you even think of algorithms, remember that quality data is your foundational block.
How we do it at Netguru
In broad strokes, here’s our process for designing AI-based products:
- AI Exploration Workshops: These workshops aim to explore the AI landscape, focusing on the needs and limitations of your particular project.
- AI Solution Design Sprint: A fast-paced brainstorming session where a potential AI solution is conceptualized.
- Data Engineering: Data is gathered, cleaned, and prepared for the upcoming phases.
- Proof-of-Concept Building: Here, a simple version of the solution is created to validate its feasibility and effectiveness.
- AI MVP Implementation: Finally, a Minimum Viable Product (MVP) is built, using AI to solve the problem at hand.
The 7 processes of machine learning
Machine learning projects generally follow a seven-step process:
- Data Gathering: The initial phase where you collect the relevant data.
- Data Pre-Processing: Data is cleaned and transformed. This can involve removing outliers or filling in missing values.
- Choose Model: You select the most suitable machine learning model for your problem.
- Train Model: The pre-processed data is used to train the selected model.
- Test Model: The model's performance is evaluated using a separate data set.
- Tune Model: Adjustments are made to the model to improve its performance.
- Prediction: The model is used for making predictions on new data.
What affects training data quality?
Data quality is influenced by:
- Process: Including operations, business rules, and communication protocols.
- People: The experience and training of the workforce handling the data.
- Tools: Labeling tools and platforms used.
Solving problems: design thinking vs machine learning
Design thinking emphasizes human-centered solutions. Its process involves steps like empathizing with your audience, defining their needs, brainstorming ideas, prototyping solutions, and testing them.
Machine Learning takes a data-driven, machine-centered approach. It starts with dissecting the needs into base components, then synthesizing a solution, brainstorming requirements, tuning the model, and validating its performance.
Both methodologies offer unique lenses through which problems can be tackled. Design thinking focuses on human needs, while machine learning concentrates on data-driven solutions.
AI model development lifecycle
The AI model development process can broadly be divided into three stages:
- Design and build: This involves data acquisition and data cleansing.
- Deploy and operationalize: At this stage, data is acquired again, annotated, and the model is validated.
- Refine and optimize: The model undergoes adjustments and optimizations for better performance and efficiency.
Understanding large language models (LLMs)
What is an LLM?
Large language models (LLM) are AI-driven powerhouses that can generate human-like text based on patterns they've learned from large datasets. The most popular is GPT, which stands for Generative Pre-trained Transformer. GPT-4 from OpenAI is the most powerful LLM as of this moment (20.10.2023), available for all to use through ChatGPT premium.
The swiss army knife of tech: LLM use-cases
LLMs can wear many hats. They can draft emails, answer questions about documents, act as a conversational agent, tutor in various subjects, translate languages, simulate video game characters, write Python code, and even give your software a natural language interface. In short, it can be the go-to tool for a multitude of tasks.
Attention: the secret LLM sauce
Why do people mention attention when discussing LLMs? The "attention mechanism" allows these models to focus on multiple parts of the input at once. Unlike their predecessors, such as LSTM or GRU, they do not rely on recurrent neural networks.
Thanks to features like multi-head attention, they can process different data types simultaneously. The architecture scales beautifully and is highly versatile, successfully tackling various natural language tasks.
The attention mechanism comes from the landmark paper “Attention is All You Need”, produced in 2017 by Google engineers who later went on to work in or found their own AI startups.
Proceed with caution: LLM limitations
LLMs can sometimes generate biased or outright false information. They may inadvertently produce sexist, racist, or homophobic responses. They also tend to falter with mathematical problems. Therefore, exercise caution when employing LLMs, especially in high-risk environments.
Cloud vs local LLMs
When it comes to hosting, you have two broad choices:
- Proprietary cloud models like GPT-4 or GPT-3.5 Turbo offer scalability and ease of use but may compromise on control and data privacy.
- Open-source local models like Llama from Meta give you greater control, privacy, and customizability but might require more work to get them to do what you want.
Each has its own set of pros and cons, so your decision should be tailored to your specific needs and constraints.
The horizon ahead: AI megatrends
The rise of co-pilots supporting human intelligence in everyday life
AI will be your new sidekick, no longer just recommending what movie to watch or product to buy, but aiding in complex daily tasks and decisions.
Code, but make it easier: genAI low-code
Low-code platforms powered by AI are making it simpler than ever to create robust applications, even if you're not a programming wizard.
The artist in the machine: generative creativity
Expect AI to eventually compose symphonies, create jaw-dropping art, and even write novels that tug at your heartstrings.
Democracy in machine learning
The tools for machine learning are becoming more accessible, letting even the tech-novices dip their toes in AI waters. Models are getting smaller and cheaper to run, and most AI research is open and available to all.
The new age of search and shopping
AI algorithms are revolutionizing how we search for information and shop online, making both experiences incredibly personalized.
The semiconductor gold rush
Companies aren't just consumers of AI; they're becoming manufacturers too, especially in the race for semiconductors. Microsoft, OpenAI, and Google among others are either working on, or already using their own AI hardware.
The global AI chess game: regulation
The European Union, the United States, and China are the biggest players grappling to set the rules of the AI game, influencing its development and adoption globally.
AI's influence on influencers
Your next Instagram influencer might just be a deepfake or a bot. In China, hundreds of thousands of deepfaked livestreamers are already raking in huge amounts of money in sales.
The classroom of tomorrow: education 2.0
AI is set to revolutionize how we learn, from personalized lesson plans to instant feedback. If you’re interested in learning more about how AI will change education, look no further than Professor Ethan Mollick, who has been studying this topic for a long time and has a rational, science-backed perspective.
Summary
Artificial Intelligence is reshaping how we solve problems, communicate, and even envision the future. More than just an innovative technology, it's a critical asset that's central to driving business success in an increasingly competitive landscape.
With a myriad of applications, from product design to natural language processing, the potential of AI is vast and still largely untapped. As we navigate the complexities and possibilities, it becomes ever more important to partner with experts who can guide us through this evolving landscape.
Netguru can help you appraise, design, and develop your AI ideas. Don't wait for the future, it’s time to accelerate!
