Scaling Compute: Training a BabyGPT
In this tutorial, we will examine how we can scale up EIR to train a baby version of the GPT model, streaming data from the FineWeb dataset for model training.
Note
This tutorial assumes you are familiar with the basics of EIR. While not required, it’s recommended to have gone through the basic tutorials first.
Note
See Streaming Data: Training with FineWeb and Streaming Data Hands-On Guide for more information on streaming data in EIR.
A - Overview
This tutorial largely follows the same approach as in Streaming Data: Training with FineWeb. Again, we will be using a WebSocket server to stream data, but this time we will be focusing on scaling compute by training a larger model.
B - Setting Up
Here’s the folder structure we’ll be working with:
eir_tutorials/i_scaling/02_scaling_compute
├── fusion.yaml
├── globals.yaml
└── output.yaml
The global config specifies basic training and below are highlighted a couple related to scaling:
globals.yaml
basic_experiment:
batch_size: 64
memory_dataset: false
n_epochs: 100
output_folder: eir_tutorials/tutorial_runs/i_scaling/02_scaling_compute
valid_size: 1024
dataloader_workers: 0
optimization:
lr: 0.0006
wd: 0.1
evaluation_checkpoint:
checkpoint_interval: 1000
n_saved_models: 1
sample_interval: 1000
data_preparation:
streaming_setup_samples: 2048
model:
compile_model: true
accelerator:
devices: auto
hardware: auto
num_nodes: 1
precision: bf16-mixed
strategy: auto
visualization_logging:
plot_skip_steps: 5000
Now, the optimization parameters are not directly related to scaling, but follow
common practice used to train large language models such as GPT2. However, we
do specify that we want to compile the model and train with a bf16-mixed precision,
which is becoming common practice for training large models. Additionally, many
options related to e.g. hardware are set to auto (which is the default),
which allows the framework to automatically select the best options for the
compute environment being used.
Note a lot of this functionality is
taking advantage of the excellent done by the folks at PyTorch
and Fabric.
You might also notice the streaming_setup_samples option there, read a
bit further down to when we are discussing the output.yaml for more
details.
For fusion, we use a simple pass-through configuration since we’re only doing sequence generation:
fusion.yaml
model_type: "pass-through"
Compared to the streaming tutorial, we can see here how we are increasing the maximum sequence length used by the model (now 512), as well as a bunch of parameters in the model configuration.
output.yaml
output_info:
output_source: ws://localhost:8000/ws
output_name: text_output
output_type: sequence
output_type_info:
max_length: 512
split_on: null
tokenizer: "bpe"
vocab_file: "fineweb_tokenizer.json"
adaptive_tokenizer_max_vocab_size: 8192
sampling_strategy_if_longer: "uniform"
min_freq: 1
model_config:
embedding_dim: 512
model_init_config:
num_layers: 4
num_heads: 8
dropout: 0.0
position: "embed"
position_dropout: 0.0
sampling_config:
generated_sequence_length: 1024
n_eval_inputs: 1
manual_inputs:
- text_output: "This movie is the most"
- text_output: "Steven"
You might also notice something new here, we are using the tokenizer option to point
to the file fineweb_tokenizer.json. This is a tokenizer file created
with the tokenizers library,
specifically the BPE tokenizer, trained on 0.5m samples from the FineWeb dataset.
You can download the tokenizer file from
this link..
Now, we can also omit this and allow EIR to train the optimizer from scratch,
this is where the streaming_setup_samples option comes in. This option controls
how many samples are collected from the streaming server to use for setup (e.g.,
training the tokenizer, estimating means for imputation, etc.). However, training
the tokenizer can take a while, so to speed things up a bit, we are using
the pre-trained tokenizer.
C - Training
Before starting training, we need to ensure our streaming server is running. The server will serve chunks of text from the FineWeb dataset. Once it’s running, we can start training:
eirtrain \
--global_configs eir_tutorials/i_scaling/02_scaling_compute/globals.yaml \
--fusion_configs eir_tutorials/i_scaling/02_scaling_compute/fusion.yaml \
--output_configs eir_tutorials/i_scaling/02_scaling_compute/output.yaml
Now, to train this, you almost certainly need a GPU. For this tutorial, the model was trained for around 2 hours on two H100 GPUs.
At iteration 500:
Auto-generated sequence at iteration 500
. Depark so recounting and a not strong pot and docal sumbers working, as some several firs that heally need to be life, we direction of blassity were of the fil has want of grow you have mast them fivice cut a park. They bystories who finder to grought and loving tework systility displication of parter in precated that do most he will now and date. I have theel saffical, bocts for a clodies in making said the freask, where weed spend this wonom of her medical diving from information by preplually promet, want here. Morld peeffers for saumber supparted take not parts of professived from and fules my grow dishing system, and said solution of initively fall be be specia can experence relatives and bets of a depect is such make the want to cocriptive done in first may side. Wherie. That presently make of the freating a storigns was do some. Macher him latching heally spender the sub cloint different brogrought’s in countle and more secon, best steased will be syprofts and breating the can a look is days their pay of a no find be has severalies of folly there will support of the currise services. In and don’t great at when must and because and counter blust. I distem make was a seased make in the solle, back here have thord that you sypants of fungancies in his do this watuated in the first tey hight be a next persity to reflument as bast serve. Most say tood and what your long and so this a collowers. Adertic insustructed by full datoring could have the some of prite. What how to but creating and was been my were a back serial trible, bas the protrolecty of manages with an how cound her mead. He we disprited systoration first a will be parts and regettly his his sub you have species work sollecty lover of requive day look sustems. By has a promet in a lature and don’t no has garmers. And for a lost and super, what a ligh
Manually generated sequence at iteration 500
This movie is the moster. The specom seffort the delious of his would professe greathending and weoch and may clast more in colute buil fined a parkett. This he back as heally on the times no her point than examer to be seasons. Hows and many det has hew surging in the butted a following carivication of food genses that we said, and the we some in mover. Brown never also discript makes of seasons he sket has a requipting out took print. The log favely and firs to the decail do. This service parted to ballings and precries the funn't was been hands distry, when meople of the sourcess. In parter and guy think on his surrish broworking homen there wancies in requalive light by fill of must of hear not be polic a don't some-time, and how farged and get travel says what cap. The more phining from look, matters have been has a genue notes of meoch of alll work teel and may requisting in your scout seace of has a surry. Whilitions of becurring to be preter in the trow firs the lard divers. Many nears that his corrates, than don't his moving and cumise his blasted as a spredition to gamilities and servical pew casually diffension. He are displetiments them tether of save the not here that want took as will be firs to the girect in my find be a states are butting in revir reach and a fame debles that pressea, it were of mon’s pritary. Chiching to gree much is no many and custer and all past was requious are nointers doing more sastry, a flogle hew will be seally reganies. He were my dive the of second may subses have to get say. Book to drach as my difficive may different of preports. Blicks of the system can be dishing on the will be decond see when a buything and considers as canially govers. This reference my bracties are parked hear here. Arso was looks in a dock will have been start of a dislet. He down a way to wating parting their don't disity
By iteration 52000, we can see improvement:
Auto-generated sequence at iteration 52000
ego. He was sort of a daunting factor in the history of the world and predicted that it would be a huge challenge for the UK to reach much well in 2013. In fact, he is now working with Coventry to give its prizes. As for his budget card in the last two years, the UK government has found the UK to reach its strongest in the world. He has been a full of supporting funding, and his prizes have been exceeded at a much lower price by others since the time of 2012 and they have fallen back to the future of the UK. Whilst many members are watching out the UK, it’s still uncertain that it is worth a small amount of luck for Coventry to sign in to the UK. In fact, he is reading about his budget card bid on 2016 and has been deprived of course of the UK’s really innovative financing technology. In addition, Summers has acquired more than 100 prizes from the UK and more than 60 prizes in 17 countries. As a blue star who does not know what to do with his future, he has signed in 2003. In 2015, Summers released their first release, Arbitrator, Max. Back to the UK, he decided on which could be a much less popular blue star when it comes to buying a new buy, and so has Max. “I know that I have a passion for getting the new buy, but I think I’m just sure how to react to the pricing price in order to stay ahead of my budget,” he says. “I have a good chance of winning this price and will have to realize that it is first time coming out of the UK, but if I make a decision, the prices will be dropped over what they get.” Summers is a great first start. For the most part, Summers are now in blue state and have been involved in the UK’s new buying specially for the past few days. “When I was in a friend my budget had to be in building strong corporate buying prices,” says Summers, who recently became the first Buyer to purchase a new buy. “I
Manually generated sequence at iteration 52000
This movie is the most trusting and exciting one for me. It was fantastic just how long I was. The characters were a popular character in my own right when I was on the road and did an incredible job of defeating my friend Evan. It wasn’t so surprising that I had a chance to defeat his friends, but when he was four years old, he really gave me a little more personal support. When I read the characters, I still get into such positively different shapes of my own. He was always a fantastic player who loved his friends. As I grew to work together more than mine. When I discovered my father, my mother was not personally biased, but my mother, who is a bit of a joke and is sure they had been both successful in making her decisions in life. She was a great friend and her team had a passion for finding the best mature moments in the movie. She was also good at playing films and succeeded in making her life the best friend of mine. With a great friend of mine, she has decided how to make things work. Her mother, a successful mother, and a grandchild, she is all the money she gets. When she says she said to me, “What would I do with that? That is the first part of my life – I have to get my mind back, so I can help myself and myself. I’m a direct memory.” She is also the mother of my father, who was never a child of my own. Her mother, Carry, lives with her mother, and her brother, Martin. She is a great friend, but she doesn’t know what she wants or needs. She also lives with her mother, and her mother specializes in making her father. She is also a good friend of mine, but she is also the most recent parent of my life. Her father, Martin (Andrew Kors) was a loving mom and a son of a mother. She is also a grandmother, and a brother and sister, Judy is a brother, and her father is also a loving mom of friends and family. S
Here’s the training curve showing our progress:
D - Complete Server Implementation
Here’s the complete implementation of our streaming server, which you can use as a reference for implementing your own:
text_streamer.py
import argparse
import os
from threading import Lock
from typing import Any
from datasets import load_dataset
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from pydantic import BaseModel
from eir.setup.streaming_data_setup.protocol import PROTOCOL_VERSION
from eir.utils.logging import get_logger
logger = get_logger(name=__name__)
app = FastAPI()
class InputInfo(BaseModel):
type: str
shape: list[int] | None = None
class OutputInfo(BaseModel):
type: str
shape: list[int] | None = None
class DatasetInfo(BaseModel):
inputs: dict[str, InputInfo]
outputs: dict[str, OutputInfo]
class ConnectionManager:
def __init__(
self,
sequence_length: int = 256,
dataset_name: str = "HuggingFaceFW/fineweb",
dataset_split: str = "train",
max_iterations: int | None = None,
):
self.active_connections: dict[WebSocket, dict] = {}
self.global_position = 0
self._position_lock = Lock()
self.dataset = None
self.sequence_length = sequence_length
self.dataset_name = dataset_name
self.dataset_split = dataset_split
self.max_iterations = max_iterations
self.validation_ids: set[str] = set()
logger.info(f"Loading dataset {dataset_name} with split {dataset_split}")
logger.info(f"Using sequence_length={self.sequence_length}")
if self.max_iterations is not None:
logger.info(f"Will terminate after {self.max_iterations} iterations")
self.dataset_iterator = None
self.load_dataset()
async def connect(self, websocket: WebSocket):
try:
await websocket.accept()
handshake_message = await websocket.receive_json()
is_not_handshake = handshake_message["type"] != "handshake"
is_incompatible_version = handshake_message["version"] != PROTOCOL_VERSION
if is_not_handshake or is_incompatible_version:
await websocket.send_json(
{
"type": "error",
"payload": {"message": "Incompatible protocol version"},
}
)
await websocket.close()
return False
worker_id = handshake_message.get("worker_id", 0)
self.active_connections[websocket] = {
"current_position": 0,
"worker_id": worker_id,
}
await websocket.send_json(
{"type": "handshake", "version": PROTOCOL_VERSION}
)
return True
except Exception as e:
logger.error(f"Error in connect: {e}")
if websocket in self.active_connections:
del self.active_connections[websocket]
return False
def disconnect(self, websocket: WebSocket):
if websocket in self.active_connections:
del self.active_connections[websocket]
async def send_personal_message(self, message: dict, websocket: WebSocket):
await websocket.send_json(message)
async def broadcast(self, message: dict):
for connection in self.active_connections:
await connection.send_json(message)
def reset(self):
if self.dataset is not None:
self.dataset_iterator = iter(self.dataset)
self.global_position = 0
if self.max_iterations is not None:
logger.info(f"Reset: Will terminate after {self.max_iterations} iterations")
def load_dataset(self):
if self.dataset is None:
name = None
path = self.dataset_name
if path == "HuggingFaceFW/fineweb":
name = "sample-10BT"
self.dataset = load_dataset(
path,
name=name,
split="train",
streaming=True,
trust_remote_code=True,
)
self.dataset_iterator = iter(self.dataset)
def get_sequence_batch(self, batch_size: int) -> list[dict[str, Any]]:
if (
self.max_iterations is not None
and self.global_position >= self.max_iterations
):
logger.info(f"Reached max iterations ({self.max_iterations}), terminating")
return []
batch = []
min_words = 20
accumulated_text = []
accumulated_words = 0
with self._position_lock:
while len(batch) < batch_size:
try:
sample = next(self.dataset_iterator)
text = sample["text"].strip()
words = text.split()
word_count = len(words)
if word_count < min_words:
continue
if accumulated_words > 0:
accumulated_text.append("<|endoftext|>")
accumulated_text.extend(words)
accumulated_words += word_count
while accumulated_words >= self.sequence_length:
chunk_words = accumulated_text[: self.sequence_length]
chunk = " ".join(chunk_words)
sample_id = f"sample_{self.global_position}"
if sample_id not in self.validation_ids:
batch.append(
{
"inputs": {"text_output": chunk},
"target_labels": {
"text_output": {"text_output": chunk}
},
"sample_id": sample_id,
}
)
accumulated_text = accumulated_text[self.sequence_length :]
accumulated_words -= self.sequence_length
self.global_position += 1
if (
self.max_iterations is not None
and self.global_position >= self.max_iterations
):
logger.info(
f"Reached max iterations ({self.max_iterations}) "
f"during batch creation"
)
return batch
if len(batch) >= batch_size:
break
except StopIteration:
logger.info("Reached end of dataset stream, restarting iterator")
self.dataset_iterator = iter(self.dataset)
continue
if accumulated_words >= min_words and len(batch) < batch_size:
chunk = " ".join(accumulated_text)
sample_id = f"sample_{self.global_position}"
if sample_id not in self.validation_ids:
batch.append(
{
"inputs": {"text_output": chunk},
"target_labels": {"text_output": {"text_output": chunk}},
"sample_id": sample_id,
}
)
self.global_position += 1
if (
self.max_iterations is not None
and self.global_position >= self.max_iterations
):
logger.info(
f"Reached max iterations ({self.max_iterations}) after "
f"adding last chunk"
)
return batch
def create_manager():
sequence_length = int(os.getenv("SEQUENCE_LENGTH", "512"))
dataset_name = os.getenv("DATASET_NAME", "HuggingFaceFW/fineweb")
dataset_split = os.getenv("DATASET_SPLIT", "train")
max_iterations_str = os.getenv("MAX_ITERATIONS")
max_iterations = int(max_iterations_str) if max_iterations_str else None
logger.info(
f"Creating ConnectionManager with sequence_length={sequence_length}, "
f"dataset_name={dataset_name}, dataset_split={dataset_split}, "
f"max_iterations={max_iterations}"
)
return ConnectionManager(
sequence_length=sequence_length,
dataset_name=dataset_name,
dataset_split=dataset_split,
max_iterations=max_iterations,
)
manager = create_manager()
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await manager.connect(websocket)
try:
while True:
data = await websocket.receive_json()
message_type = data.get("type")
if message_type == "getInfo":
dataset_info = DatasetInfo(
inputs={},
outputs={
"text_output": OutputInfo(type="sequence"),
},
)
await manager.send_personal_message(
message={"type": "info", "payload": dataset_info.model_dump()},
websocket=websocket,
)
elif message_type == "getData":
batch_size = data.get("payload", {}).get("batch_size", 32)
batch = manager.get_sequence_batch(batch_size=batch_size)
if not batch:
await manager.send_personal_message(
message={"type": "data", "payload": ["terminate"]},
websocket=websocket,
)
break
await manager.send_personal_message(
message={"type": "data", "payload": batch}, websocket=websocket
)
elif message_type == "setValidationIds":
validation_ids = data.get("payload", {}).get("validation_ids", [])
manager.validation_ids = set(validation_ids)
await manager.send_personal_message(
message={
"type": "validationIdsConfirmation",
"payload": {
"message": f"Received {len(validation_ids)} validation IDs"
},
},
websocket=websocket,
)
elif message_type == "reset":
manager.reset()
await manager.send_personal_message(
message={
"type": "resetConfirmation",
"payload": {"message": "Reset successful"},
},
websocket=websocket,
)
await manager.broadcast(
message={
"type": "reset",
"payload": {"message": "Reset command received"},
}
)
elif message_type == "status":
status_data = {
"active_connections": len(manager.active_connections),
"current_position": manager.global_position,
"validation_ids_count": len(manager.validation_ids),
}
if manager.max_iterations is not None:
status_data["max_iterations"] = manager.max_iterations
status_data["remaining_iterations"] = max(
0, manager.max_iterations - manager.global_position
)
await manager.send_personal_message(
message={"type": "status", "payload": status_data},
websocket=websocket,
)
elif message_type == "heartbeat":
await manager.send_personal_message(
message={"type": "heartbeat"}, websocket=websocket
)
except WebSocketDisconnect:
manager.disconnect(websocket)
finally:
manager.disconnect(websocket)
def main():
parser = argparse.ArgumentParser(description="Run the data streaming server")
parser.add_argument(
"--host", type=str, default="0.0.0.0", help="Host to run the server on"
)
parser.add_argument(
"--port", type=int, default=8000, help="Port to run the server on"
)
parser.add_argument(
"--max-iterations",
type=int,
default=None,
help="Maximum number of iterations before terminating (default: no limit)",
)
args = parser.parse_args()
if args.max_iterations is not None:
os.environ["MAX_ITERATIONS"] = str(args.max_iterations)
import uvicorn
uvicorn.run(app, host=args.host, port=args.port, ws_ping_timeout=3600)
if __name__ == "__main__":
main()
E - Supervised Fine-Tuning from a Pretrained Model
In this section, we will explore how to perform supervised fine-tuning (SFT) using a pretrained model. This approach allows us to leverage the knowledge already captured in a pretrained model and adapt it to a new specific task.
For this example, we’ll use the model we trained in the previous section as our pretrained checkpoint and fine-tune it on the Alpaca dataset, which contains instruction-following examples.
Training
Note
If you look at the code in the text_streamer.py file,
you might notice that the dataset to use is grabbed from a
DATASET_NAME environment variable. To use the Alpaca dataset,
start the server with the following command:
DATASET_NAME=tatsu-lab/alpaca python text_streamer.py.
Before starting the fine-tuning process, we need to set up our command with the path to the pretrained model:
eirtrain \
--global_configs eir_tutorials/i_scaling/02_scaling_compute/globals.yaml \
--fusion_configs eir_tutorials/i_scaling/02_scaling_compute/fusion.yaml \
--output_configs eir_tutorials/i_scaling/02_scaling_compute/output.yaml \
--globals.basic_experiment.output_folder=eir_tutorials/tutorial_runs/i_scaling/02_scaling_compute_sft_pretrained/ \
--globals.model.pretrained_checkpoint=eir_tutorials/tutorial_runs/i_scaling/02_scaling_compute/saved_models/02_scaling_compute_checkpoint_88000_perf-average=-0.7236.pt
Note the important parameters in the command above:
We’re specifying a new output folder for our fine-tuned model
We’re providing the path to our pretrained checkpoint via the
pretrained_checkpointparameter
This time, our training should be much faster since we’re starting from a pretrained model. Let’s look at the training curve:
Let’s examine the output at different stages of training. At iteration 500:
Auto-generated response at iteration 500
esponse: My cousins are a friendly and hard worker, who is a major gifted supervisor with a high level of compassion. She has a wealth of experience, innovation, and career opportunities that are easily accessible and profitable. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Given a sentence, explain why the sentence is more concise. ### Input: She was happy with her job. ### Response: She was delighted with her job. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Compose a poem about an individual who is going through a series of struggles to build a successful business ### Input: She was so big and smart, she decided to build a successful business ### Response: She was so big and smart, she decided to build a successful business in order to build a successful business. She built a successful business and gave it a try, she felt the need to finish managing decisions and creating a successful business plan. She had always wanted to build a successful business, so she needed help and confidence to make the best decisions. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Summarize the following article, what it contains about deforestation. ### Input: Deforestation in Africa has become an issue of many regions, from habitats to land areas. Extending our livelihood of cultural experiences, such as natural resources like wildlife, habitat loss, and soil production, have become increasingly imp
The initial outputs might be somewhat incoherent as the model is still learning the instruction format. By iteration 12000, we can see significant improvement:
Auto-generated response at iteration 12000
instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: List three countries with longest-human hostility. ### Response: Countries with longest-human hostility continue to lose frequency in the war today, which leads to higher taxes and more job losses. This is not a post-anonymity tone, as existing literary devices can be used to create a world where each person has the true ability to leave their home or workplace. <|endoftext|> Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: Describe the benefits of using natural language processing (NLP) technologies. ### Response: Using natural language processing (NLP) technology is beneficial because it can help inform decisions about large amounts of data and can lead to increased efficiency in customer service operations. NLP is a branch of artificial intelligence that is tailored to the desired situation and technologies of the people living in major cities and large states around the world. This approach has been used to improve the quality of life in the world and to improve public ownership. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Add two sentences to this paragraph to make it more interesting ### Input: The population of Africa in 2019 was greater than that population. In 2018, out of the 112,435 people lived around 14,000 individuals, who were most excited to have a job. This impact, however, has changed in many ways due to adversity, leaving increasingly diverse communities needing to continue to take steps. ### Response: The particular impact of technology in recent years have been immense. For
Now the model is starting to generate coherent responses that follow the instruction format, demonstrating the effectiveness of supervised fine-tuning.
F - Deploying the Model
In this section, we’ll demonstrate how to deploy our fine-tuned model as a web service and interact with it using HTTP requests.
Starting the Web Service
To serve the model, we use the eirserve command:
eirserve \
--device cuda \
--model-path eir_tutorials/tutorial_runs/i_scaling/02_scaling_compute_sft_pretrained/saved_models/02_scaling_compute_sft_pretrained_checkpoint_31000_perf-average=0.7754.pt
This command initiates a web service that listens for incoming requests on the default port (8000). Note that we’re specifying:
The GPU device for inference
The path to our fine-tuned model
Sending Requests
With the server running, we can now send inference requests to our model. Here are two approaches to interact with the deployed model:
Python Example
Here’s a Python function for sending requests to the model:
python_request_example_module.py
import requests
def send_request(url: str, payload: list[dict]) -> list[dict]:
response = requests.post(url, json=payload)
response.raise_for_status()
return response.json()
payload = [
{
"text_output": "### Instruction: Write a short story about a robot "
"learning to feel emotions. ### Response:"
},
]
response = send_request(url="http://localhost:8000/predict", payload=payload)
print(response)
When running this code, we get the following response:
python_request_example.json
{
"result": [
{
"text_output": "### Instruction: Write a short story about a robot learning to feel emotions. ### Response: Once upon a time, there was a little girl who loved the small robot discovering her innate furniture. Even in her own small town, she knew she was about to embark on a journey. The robot granted her moment of purpose and encountered many strange creatures, determined not only to discover her hidden faces, but about a mysterious sign of the creature. The robot granted them a magical power and often explained it, coming in a lot of wonder to her new friends. The robot greeted her for help and the robot granted her one of their magical powers and invited them to join off in the darkness of the forest. From that day forward, the robot lived a life of abundance and protection in the knowledge and power of friendships. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Edit this sentence to improve it. ### Input: I had a great time at the party. ### Response: I had an exceptional time at the party. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Create a function that takes an array of numbers and sorts its addition in ascending order. ### Input: [5, 2, 9, 7, 4] ### Response: def sort(arr): add = 0 for i in range(len(arr)): lower_sorted(arr) # sort in length else: # create open an array of numbers loud = [] for i in range(lower_sorted(arr): if i % 3 == 0: median += 1 else: median=x^2 # Print all prime numbers print(\"sorted array\") # Output: 3 7, 2, 5 <|endoftext|> Below "
}
]
}
Note
Notice how in the response, we often end up with multiple instruction-answer pairs. This is because how the data is structured for the model in the streaming server, where multiple samples are aggregated together into a single sequence. Optimally, we might want to change this for the SFT part to only feed one sample at a time, or build some trimming functionality into our response handling.
Bash Example
Alternatively, you can use a simple curl command from bash:
bash_request_example_module.sh
curl -X POST \
"http://localhost:8000/predict" \
-H "accept: application/json" \
-H "Content-Type: application/json" \
-d '[{"text_output": "### Instruction: Explain three ways to reduce carbon
emissions. ### Response:"}]
'
This produces a similar response:
bash_request_example.json
{
"detail": [
{
"type": "json_invalid",
"loc": [
"body",
70
],
"msg": "JSON decode error",
"input": {},
"ctx": {
"error": "Invalid control character at"
}
}
]
}
Analyzing Responses
Let’s look at some example responses from our model for various instructions:
predictions.json
[
{
"request": [
{
"text_output": "### Instruction: Generate a list of five healthy breakfast ideas. ### Response:"
},
{
"text_output": "### Instruction: Explain quantum computing in simple terms. ### Response:"
},
{
"text_output": "### Instruction: Write a short poem about artificial intelligence. ### Response:"
},
{
"text_output": "### Instruction: Give me three tips for improving time management. ### Response:"
},
{
"text_output": "### Instruction: Describe the process of photosynthesis. ### Response:"
}
],
"response": {
"result": [
{
"text_output": "### Instruction: Generate a list of five healthy breakfast ideas. ### Response: 1. Overnight oats with fruit and nuts 2. Egg frittata with vegetables 3. Avocado toast with a boiled egg 4. Smoothie bowl with bananas 5. Bananas with the ium into a berry cup 6. Apples sliced in a warm maple smoothie 7. Homemade Protein smoothie 8. Oatmeal 9. Cahochili 10. Grilled Vegetable Soup <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Compose a tweet to celebrate this song. ### Input: \"We blank you this song!\" ### Response: Oh, this song is terrible! I don't know you can enjoy this song. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Rewrite this sentence using a vocabulary word from the provided list. ### Input: She ate a lullaby to the teacher. ### Response: She ate too much food. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Compare two different advertisements ### Input: Razer and Emma ### Response: Advertisements provide an invaluable look at the concept of emmonym for recent generations. This bold lack of communication with family and friends has serious consequences when it comes to marketing. However, some of the challenges these commercials expect to recognize: having a positive attitude and finding communal good for someone, and how well are clarifying it. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context"
},
{
"text_output": "### Instruction: Explain quantum computing in simple terms. ### Response: Quantum computing is a branch of computing technology that involves the fixing and solving of quantum patterns, quantum simulations and quantum-leveable phenomena. These qubits are used to process real-world data that is composed of multiple bits of data, such as brain weights, biases, and results. Quantum computing can be used to perform complex computations, particularly when data is not possessed in quantum computing. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Classify this sentence as a sentence fragment, dependent clause, independent clause, etc. ### Input: The first sentence of the book. ### Response: independent clause <|endoftext|> Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: Give me an example of a sentence that starts with \"light\" and is in 1:3; this is independent clause in \"listen\". ### Response: Light is like the water, bolfage of light, which is filled with heart. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Give a love smile from the following simile. ### Input: Camaraderie ### Response: Hey camaraderie, just take a break! Together you're taking into life and making similar changes, can you forgive a best? <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Given a list of integers, find the first missing number in this ar"
},
{
"text_output": "### Instruction: Write a short poem about artificial intelligence. ### Response: AI is an incredible fact that the computer was known for its practicalges and systems, both often made from a variety of materials, and even though the information is programmed to be programmed. AI is used in a wide variety of fields, such as robotics, healthcare, automation, and manual processing. It is also used in industrious applications such as healthcare, finance, manufacturing, and transportation. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Create a sample tweet to advertise the following product ### Input: A milkshake ### Response: What greens are in the milkshake, from the chimpanzees and z-supplements? Check out the concept of Milkshake Shake Sulfur to your alien mother. With Milkshake, the milk spreads milk, dry avocado, and hot sauces that have a strong affine that transparent and thoughtful. This is particularly notable in the case of the Allied forces and is considered one of the biggest military commands of all countries and affected by Middle’s success. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Give three uses of the given words ### Input: Abe, Chef, Vegetable ### Response: - Both uses cooking and crafting techniques to create meaningful flavors. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Give me a list of 5 adjectives that best describe a rose ### Input: Oak Tree ### Response: Vincent, m"
},
{
"text_output": "### Instruction: Give me three tips for improving time management. ### Response: Three tips for improving time management are: 1) Take a break and rest, 2) Make use of a timer or tools, and plan ahead to make sure you are staying on track. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Change the adverb to an adjective in future tense. ### Input: I will take the calculater this month. ### Response: I will have to Take the Calculater this month. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Find the most important takeaway from the following article. ### Input: An article entitled \"Does Technology Make Us More than Cost Educate?\" ### Response: The most important takeaway from the article is that Technology is allowing us to make the more time offered and to enjoy the pressing of events when it is no longer beneficial to the public. While technology can provide information about content delivery, it is also important to be mindful of one's impact on the way we interact with each other and connect with those close to us. Technology has the power to offer something that enabled us to stay connected even while we are being explored by broadbent devices. Furthermore, it allows us to communicate with one another acros time, travel and and lives, and gain insight from different sources of information. In conclusion, having an effective onboard is key to any successful business, and that's what it surpasses the market's omnism and convenience. <|endoftext|> Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: Generate a list of five w"
},
{
"text_output": "### Instruction: Describe the process of photosynthesis. ### Response: Photosynthesis involves photosynthesis (PHC)/2. It begins with the light at the displacement level, where it is absorbed by plants and oxygen, which are converted into chemical energy stored in oxygen as waj from the sun. The chlorophyllum is responsible for absorbing water and releasing mesh starters, which are used to supplement the existing flow and production. The pH is then broken, evaluate, or calculate the amount of energy required for operations of the system in the world today. <|endoftext|> Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: Compare the structure of mascendors with their flight flying before mousin. ### Response: Mascendors are often seen on planes like jets, garages, and booths where they are all connected. They usually involve millions of dolphins in a wide range of spacecrafts, all the including mouse, open appliances, and playdating contacts, instead of those that rely on radio or forage for help. Both can benefit from the potential of masking tips and also be an example of how to maintain good eating habits in terms of stress reduction. <|endoftext|> Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: What is the period of the sun's orbit around the Milk Changs? ### Response: The period of the sun's orbit is generally dominated by Glacius emigration. This can mean the year is also highly regional, due to a warming current; however, Glacius is faster than Glacius; it takes less storage of essential elements and allows for more complex regulation. <|endoftext|> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Classify the fol"
}
]
}
}
]
As we can see, the model is able to generate appropriate responses for different types of instructions, from generating creative content to providing explanations and advice.
G - Conclusion
In this tutorial,
we’ve explored how to scale up EIR to
train a baby version of the GPT model,
stream data for training,
and perform supervised fine-tuning using a pretrained checkpoint.
We’ve also learned how to
deploy the model as a web service and
interact with it using HTTP requests.
Thank you for reading!