> For the complete documentation index, see [llms.txt](https://aadesh-kumar.gitbook.io/generative-ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://aadesh-kumar.gitbook.io/generative-ai/generative-ai.md). # Generative AI Generative AI refers to a class of AI technologies that is capable of generating various forms of content, including but not limited to text, images, audio, and video. These AI systems can generate new content, \ based on their training data and input parameters, which usually include text prompts but can also involve other forms of input such as images. Lets Understand that previous architecture that will be used before the LLM models | Task | Popular Architecture(s) Before Modern GenAI | Notes | | -------------------- | ------------------------------------------- | ------------------------------------------------------- | | **Text Generation** | Seq2Seq (RNN / LSTM based) | Encoder-decoder RNNs with attention | | **Image Generation** | DCGAN (Deep Convolutional GAN) | Early GANs generating realistic images | | **Video Generation** | 3D CNNs, Recurrent Video GANs (VGAN) | Frame-by-frame generation, limited quality | | **Speech / Audio** | WaveNet, RNN-based autoencoders | High-quality speech synthesis | | **Image-to-Image** | Pix2Pix, CycleGAN | Conditional GANs for style transfer, domain translation | ***Type of the Generative AI Models*** 1. **GAN (Generative adversarial networks)** The generator network generates an image from a latent vector (noise vector) and is a type of unsupervised \ machine learning. Latent space is a compressed representation of some data.
Initial Generated Images
After learning Images
final Learning Image
GAN are Working on the **MinMax Loss** 2. **VAE(variational autoencoders)** A **Variational Autoencoder** is a type of **generative model** that learns to represent data in a **compressed latent space**, and then generate **new, similar data** from that space. Core mapping between the data Hidden Markov model -> Statistical method Latent Space -> It is refer to the space where all the information and the hidden feature will be store by the encoder during the training. And we will use that latent space to make the similar kind of the data

Encoder - decoder

They are 3 Type: - VAE \ 1\. no memory \ 2\. Using only the latent space for the generation.\ 3\. Zero Control Over Generation. Beta - VAE 1. We can control over the image generation in beta vae(like color, shape) 2. We providing the some information in the architecture Conditional VAE 1. In this we are providing the some kind of information during the generation **Tranformer Architecture**

Transformer Architecture

**Self-attention** is a mechanism used to capture the contextual meaning between words in a sentence. It helps the model understand how closely related different words are to each other, regardless of their position in the sentence. **Example:**\ In the sentence:\ **"The animal didn't cross the street because it was too tired."**\ The word **"it"** refers to **"the animal"**.\ Self-attention allows the model to make this connection by assigning higher attention scores between **"it"** and **"animal"**, helping it understand the context better. Similarly, the **Multi head Attention** is used for the getting the contextual embedding of the multiple words. **Feed forward Neural Network:** - It is used for the matrix multiplication and obtaining the best weights value. **Positinal Encoding**
| Feature | **LLM (Large Language Model)** | **Agentic AI** | | -------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------ | | **Core Purpose** | Generate language (text, code, etc.) based on a prompt | Solve goals **autonomously** using tools, memory, and reasoning | | **Behavior** | Passive — responds to prompts one at a time | Active — plans, reasons, and takes actions without being prompted | | **Memory** | Stateless or limited context window (unless extended via memory) | Often includes **long-term memory** and **working memory** | | **Tools / Actions** | Doesn't take actions unless asked | Can **use tools, APIs, search the web, call functions** | | **Goal Orientation** | No goal — just predicts next token | Goal-driven — executes tasks, refines plans, self-corrects | | **Architecture** | Pure LLM (e.g., GPT, Claude, LLaMA) | LLM + Orchestrator + Tool use + Memory + Feedback loop | | **Example Behavior** | "Write an email." → Writes it once | "Book my travel." → Searches, books, confirms, retries if failed | | **Autonomy** | Human-in-the-loop | **Can run in loops** with minimal or no human intervention | | **Examples** | ChatGPT (standard mode) |

AutoGPT, Devin, OpenAI's Assistants API, Microsoft AutoGen

| ***LCM (Long Context Model )*** \ \ In simple term the LCM is the process to store the large amount of information/context generally the older model are only able to store the 4k to 32k but now a days the models like GPT4o is designed for much longer contexts (tens or hundreds of thousands of tokens). \------------------------------------------------------------------------------- **Language Composition Models: -** aim to build language understanding **the way humans do** — by combining smaller ideas into more complex ones. They're useful where precise, structured, or logical understanding is needed. \------------------------------------------------------------------------------- **Quantization: -** In machine learning, model quantization is a technique to reduce the size and computational cost of machine learning models, particularly large language models (LLMs), by converting high-precision floating-point numbers (like 32-bit or 16-bit) to lower-precision representations, typically 8-bit integers (INT8) or even lower. Example :- When we try to use a trained model in real-time applications, such as on a Raspberry Pi, we face limitations like small storage and limited computational resources. In such cases, quantized models come into play. Quantized models have similar capabilities to larger models but with significantly reduced size and computational requirements. As a result, the performance of the quantized model is comparable to that of the original full-sized model.

Quantize Model process

**Model Context Protocol** MCP is used for designing the Complex AI agent workflow simple. it is the open source protocol that standardizes how applications provide context to LLMs. Example:- Github MCP server that will be used for doing the Github operation in natural language. The user will simply chat with the Github for the repo create, pull, merge and other operation. The user will interact with the LLM model and then LLM model will identify that which model is used for the user task. then LLM model format the input and pass to that MCP tool for completing the task. \ **General architecture**

MCP server

**Stable Diffusion** The stable Diffusion model is a generative AI model that will be using for generating the realistic image. based on the text and Image + text. The Stable Diffusion is using the Latent space for making the image. The latent space is hidden space where the hidden feature is store during the training period. during the inference time the latent space is used for making the Image generation. As a diffusion model, Stable Diffusion differs from many other image generation models. In principle, diffusion models use Gaussian noise to encode an image. Then, they use a noise predictor together with a reverse diffusion process to recreate the image. Apart from having the technical differences of a diffusion model, Stable Diffusion is unique in that it doesn’t use the pixel space of the image. Instead, it uses a reduced-definition latent space. The reason for this is that a color image with 512x512 resolution has 786,432 possible values. By comparison, Stable Diffusion uses a compressed image that is 48 times smaller at 16,384 values. Then use the reverse diffusion model for recreate the image.
#### Variational autoencoder The variational autoencoder consists of a separate encoder and decoder. The encoder compresses the 512x512 pixel image into a smaller 64x64 model in latent space that's easier to manipulate. The decoder restores the model from latent space into a full-size 512x512 pixel image. #### Forward diffusion Forward diffusion progressively adds Gaussian noise to an image until all that remains is random noise. It’s not possible to identify what the image was from the final noisy image. During training, all images go through this process. Forward diffusion is not further used except when performing an image-to-image conversion. #### Reverse diffusion This process is essentially a parameterized process that iteratively undoes the forward diffusion. For example, you could train the model with only two images, like a cat and a dog. If you did, the reverse process would drift towards either a cat or dog and nothing in between. In practice, model training involves billions of images and uses prompts to create unique images. #### Noise predictor (U-Net) A noise predictor is key for denoising images. Stable Diffusion uses a U-Net model to perform this. U-Net models are convolutional [neural networks](https://aws.amazon.com/what-is/neural-network/) originally developed for image segmentation in biomedicine. In particular, Stable Diffusion uses the Residual Neural Network (ResNet) model developed for computer vision. The noise predictor estimates the amount of noise in the latent space and subtracts this from the image. It repeats this process a specified number of times, reducing noise according to user-specified steps. The noise predictor is sensitive to conditioning prompts that help determine the final image. #### Text conditioning The most common form of conditioning is text prompts. A CLIP tokenizer analyzes each word in a textual prompt and embeds this data into a 768-value vector. You can use up to 75 tokens in a prompt. Stable Diffusion feeds these prompts from the text encoder to the U-Net noise predictor using a text transformer. By setting the seed to a random number generator, you can generate different images in the latent space. Stable Diffusion represents a notable improvement in text-to-image model generation. It’s broadly available and needs significantly less processing power than many other text-to-image models. Its capabilities include text-to-image, image-to-image, graphic artwork, image editing, and video creation. #### Text-to-image generation This is the most common way people use Stable Diffusion. Stable Diffusion generates an image using a textual prompt. You can create different images by adjusting the seed number for the random generator or changing the denoising schedule for different effects. #### Image-to-image generation Using an input image and text prompt, you can create images based on an input image. A typical case would be to use a sketch and a suitable prompt. #### Creation of graphics, artwork and logos Using a selection of prompts, it’s possible to create artwork, graphics and logos in a wide variety of styles. Naturally, it's not possible to predetermine the output, although you can guide logo creation using a sketch. #### Image editing and retouching You can use Stable Diffusion to edit and retouch photos. Using AI Editor, load an image and use an eraser brush to mask the area you want to edit. Then, by generating a prompt defining what you want to achieve, edit or inpaint the picture. For example, you can repair old photos, remove objects from pictures, change subject features, and add new elements to the picture. #### Video creation Using features such as Deforum from GitHub, it’s possible for you to create short video clips and animations with Stable Diffusion. Another application is to add different styles to a movie. It’s also possible for you to animate photos by creating an impression of motion, like with flowing water.