Hướng dẫn Xây dựng Hệ Thống RAG sử dụng OpenAI với PDF

Tổng Quan

Hướng dẫn cách xây dựng một hệ thống RAG (Retrieval-Augmented Generation) sử dụng:

• PDF cookbook (sách nấu ăn định dạng PDF)

• OCR để trích xuất văn bản từ ảnh

• OpenAI GPT để trích xuất thông tin có cấu trúc

• Embeddings để tìm kiếm theo ngữ nghĩa

• RAG để trả lời câu hỏi từ người dùng dựa trên thông tin đã trích xuất

I. Cài đặt

# Import the userdata module from Google Colab
from google.colab import userdata
# Retrieve the API key stored under 'genai_course' from Colab's userdata
api_key = userdata.get('genai_course')

# Mount the drive
from google.colab import drive
drive.mount('/content/drive')

# Change directory to this folder
%cd /content/drive/MyDrive/GenAI/RAG/RAG with OpenAI

II.  Thực hiện OCR và chuyển đổi thành hình ảnh

# Install the pdf2image library for converting PDF files to images
!pip install pdf2image
# Install the poppler-utils package, required by pdf2image to work with PDF files
!apt-get install -y poppler-utils

from pdf2image import convert_from_path
import os

# Hàm chuyển PDF thành ảnh và lưu đường dẫn ảnh
def pdf_to_images(pdf_path, output_folder):
    # Tạo thư mục lưu ảnh nếu chưa tồn tại
  if not os.path.exists(output_folder):
    os.makedirs(output_folder)

  # Convert PDF into images
  images = convert_from_path(pdf_path) # Convert each page of the PDF to an image
  image_paths = []

  # Save images and store their paths
  for i, image in enumerate(images):
    image_path = os.path.join(output_folder, f"page{i+1}.jpg") # Generate the image file path
    image.save(image_path, "JPEG") # Save the image as a JPEG file
    image_paths.append(image_path) # Append the image path to the list

  return image_paths # Return the list of image paths

# Xác định đường dẫn đến file PDF và thư mục để lưu ảnh đầu ra
pdf_path = "Things mother used to make.pdf"
output_folder = "images"

# Chuyển đổi PDF thành các ảnh và lưu các đường dẫn ảnh
image_paths = pdf_to_images(pdf_path, output_folder)

# Install the openAI library
!pip install openai

# Import the libraries
from openai import OpenAI
import base64

# Set up connection to OpenAI API
client = OpenAI(
    api_key=api_key, # Use the provided API key for authentication
)
# Specify the model to be used
model = "gpt-4o-mini"

# Read and encode one image
image_path = "images/page23.jpg" # Path to the image to be encoded

# Encode the image in base64 and decode to string
with open(image_path, "rb") as image_file:
  image_data = base64.b64encode(image_file.read()).decode('utf-8')
image_data

# Define the system prompt
system_prompt = """
Please analyze the content of this image and extract any related recipe information.
"""

# Call the OpenAI API use the chat completion method
response = client.chat.completions.create(
    model = model,
    messages = [
        # Provide the system prompt
        {"role": "system", "content": system_prompt},

        # The user message contains both the text and image URL / path
        {"role": "user", "content": [
            "This is the imsage from the recipe page.",
            {"type": "image_url",
             "image_url": {"url": f"data:image/jpeg;base64,{image_data}",
                           "detail": "low"}}
        ]}
    ]
)

# Retrieve the content
gpt_response = response.choices[0].message.content

from IPython.display import Markdown, display

# Display the GPT response as Markdown
display(Markdown(gpt_response))

# Define a function to get the GPT response and display it in Markdown
def get_gpt_response():
  gpt_response = response.choices[0].message.content # Extract the response content from the API response
  return display(Markdown(gpt_response)) # Display the response as Markdown

# Call the function to display the GPT response
get_gpt_response()

Here are the recipes extracted from the image:

Bannocks

Ingredients:

• 1 Cupful of Thick Sour Milk
• ½ Cupful of Sugar
• 2 Cupfuls of Flour
• ½ Cupful of Indian Meal
• 1 Teaspoonful of Soda
• A pinch of Salt

Instructions:

1. Make the mixture stiff enough to drop from a spoon.
2. Drop mixture, size of a walnut, into boiling fat.
3. Serve warm with maple syrup.

Boston Brown Bread

Ingredients:

• 1 Cupful of Rye Meal
• 1 Cupful of Sour Milk
• 1 Cupful of Graham Meal
• 1 Cupful of Molasses
• 1 Cupful of Flour
• ½ Teaspoonful of Indian Meal
• 1 Cupful of Sweet Milk
• 1 Heaping Teaspoonful of Soda

Instructions:

1. Stir the meals and salt together.
2. Beat the soda into the molasses until it foams; add sour milk, mix well, and pour into a tin pan which has been well greased.
3. If you have no brown-bread steamer, bake in the oven.

Feel free to let me know if you need any more help!

# Define improved system prompt
system_prompt2 = """
Please analyze the content of this image and extract any related recipe information into structure components.
Specifically, extra the recipe title, list of ingredients, step by step instructions, cuisine type, dish type, any relevant tags or metadata.
The output must be formatted in a way suited for embedding in a Retrieval Augmented Generation (RAG) system.
"""

# Call the api to extract the information
response = client.chat.completions.create(
    model = model,
    messages = [
        # Provide the system prompt
        {"role": "system", "content": system_prompt2},

         # The user message contains both the text and image URL / path
        {"role": "user", "content": [
            "This is the image from the recipe page",
            {"type": "image_url",
             "image_url": {"url": f"data:image/jpeg;base64,{image_data}",
                           "detail": "low"}}
        ]}
    ],
    temperature = 0, # Set the temperature to 0 for deterministic output
)

# Print the info from the page with the improved prompt
get_gpt_response()

Here’s the structured information extracted from the recipe image:

Recipe Title

Breads

Ingredients

Bannocks

• 1 Cupful of Thick Sour Milk
• ½ Cupful of Sugar
• 2 Cupfuls of Flour
• ½ Cupful of Indian Meal
• 1 Teaspoonful of Soda
• A pinch of Salt

Boston Brown Bread

• 1 Cupful of Rye Meal
• 1 Cupful of Graham Meal
• 1 Cupful of Molasses
• 1 Cupful of Flour
• 1 Cupful of Sweet Milk
• 1 Cupful of Sour Milk
• ½ Teaspoonful of Salt
• 1 Teaspoonful of Soda
• 1 Heaping Teaspoonful of Baking Powder

Step-by-Step Instructions

Bannocks

1. Make the mixture stiff enough to drop from a spoon.
2. Drop mixture, size of a walnut, into boiling fat.
3. Serve warm, with maple syrup.

Boston Brown Bread

1. Stir the meals and salt together.
2. Beat the soda into the molasses until it foams; add sour milk, mix well, and pour into a tin pan which has been well greased.
3. If you have no brown-bread steamer, use a regular oven.

Cuisine Type

Traditional American

Dish Type

Breads

Relevant Tags/Metadata

• Quick Bread
• Breakfast
• Comfort Food
• Homemade

This format is suitable for embedding in a Retrieval Augmented Generation (RAG) system.

# Extract information about all of the images/recipes
extracted_recipes = []

for image_path in image_paths:
  print(f"Processing image {image_path}")

  # Reading and decoding the image
  with open(image_path, "rb") as image_file:
    image_data = base64.b64encode(image_file.read()).decode("utf-8") # Encode the image to base64 format

  # Call the API to extract the information
  response = client.chat.completions.create(
      model = model,
      messages = [
          # Provide system prompt for guidance
          {"role": "system", "content": system_prompt2},

          # The user message contains both the text and image URL / path
          {"role": "user", "content": [
              "This is the image from the recipe page", # Context for the image
              {"type": "image_url",
              "image_url": {"url": f"data:image/jpeg;base64,{image_data}", # Provide the base64 image
                            "detail": "low"}}
          ]}
      ],
      temperature = 0, # Set the temperature to 0 for deterministic output
  )

  # Extract the content and store it
  gpt_response = response.choices[0].message.content # Get the response content
  extracted_recipes.append({"image_path": image_path, "recipe_info": gpt_response}) # Store the path and extracted info
  print(f"Extracted information for {image_path}:\n{gpt_response}\n") # Print the extracted information for review

Streaming output truncated to the last 5000 lines. ### Cuisine Type American ### Dish Type Dessert ### Relevant Tags - Baking - Fruit Dessert - Traditional --- ### Recipe Title Quick Graham Bread ### Ingredients - 1 Pint of Graham Meal - 1 Cup of Soda - ½ Cup of Molasses - 1 Cup of Sour Milk - ½ Teaspoon of Salt ### Instructions

Processing image images/page136.jpg Extracted information for images/page136.jpg: I'm unable to analyze the content of the image directly. If you can provide the text or details from the recipe, I can help you structure that information into the desired format.

# Filter out non-recipe content based on key recipe-related terms
filtered_recipes = []

for recipe in extracted_recipes:
  # Check if the extracted content contains any key recipe-related terms
  if any(keyword in recipe["recipe_info"].lower() for keyword in ["ingredients",
                                                                  "instructions",
                                                                  "recipe title"]):
     # If it does, add it to the filtered list
    filtered_recipes.append(recipe)

  # Print a message for non-recipe content
  else:
    print(f"Skipping recipe: {recipe['image_path']}")

Skipping recipe: images/page1.jpg Skipping recipe: images/page2.jpg Skipping recipe: images/page3.jpg Skipping recipe: images/page4.jpg Skipping recipe: images/page5.jpg Skipping recipe: images/page6.jpg Skipping recipe: images/page8.jpg Skipping recipe: images/page10.jpg Skipping recipe: images/page11.jpg Skipping recipe: images/page12.jpg Skipping recipe: images/page20.jpg Skipping recipe: images/page21.jpg

# import json library
import json

# Define the output file path
output_file = "recipe_info.json"

# Write the filtered list to a json file
with open(output_file, "w") as json_file:
  json.dump(filtered_recipes, json_file, indent = 4)

III. Embeddings

Embedding là cách chuyển văn bản thành một vector số học (danh sách các số thực), sao cho mô hình có thể hiểu được ngữ nghĩa và ngữ cảnh của văn bản đó.

Ví dụ:

• Câu "Tôi thích ăn bánh mì" và "Tôi yêu bánh mì" sẽ có embedding rất gần nhau.

• Trong khi "Tôi đang học lập trình" sẽ có vector khác xa hơn.

# import libraries
import numpy as np

# Load the filtered recipes
with open("recipe_info.json", "r") as json_file:
  filtered_recipes = json.load(json_file)

# Generate embeddings for each recipe
recipe_texts = [recipe["recipe_info"] for recipe in filtered_recipes] # Extract the text content of each recipe

# Call the API to generate embeddings for the recipe texts
embedding_response = client.embeddings.create(
    input = recipe_texts, # Provide the list of recipe texts as input
    model = "text-embedding-3-large" # Specify the embedding model to use
)

• Mỗi mục trong file đại diện cho một công thức, bao gồm:

  • "image_path": đường dẫn ảnh gốc từ PDF.

  • "recipe_info": văn bản đã được GPT trích xuất và định dạng lại từ ảnh.

• Tạo danh sách recipe_texts chứa toàn bộ văn bản "recipe_info" từ mỗi công thức.

• Đây chính là đầu vào để tạo embedding.

• client.embeddings.create: gọi API của OpenAI để tạo embedding.

• input: danh sách văn bản cần vector hóa (các công thức nấu ăn).

• model: sử dụng model "text-embedding-3-large" — một model mạnh và chính xác để hiểu ngữ nghĩa văn bản.

# Extract the embeddings
embeddings = [data.embedding for data in embedding_response.data]
embeddings

• embedding_response.data là danh sách các embedding được tạo.

• Mỗi data.embedding là một vector (ví dụ: [0.124, -0.552, 0.789, ...]).

• Bạn lưu lại tất cả các embedding thành danh sách embeddings.

[[-0.018192430958151817, -0.03411807492375374, -0.0201831366866827, -0.015010208822786808, 0.026213375851511955, -0.035687390714883804, -0.016187194734811783, 0.0008405099506489933, -0.015751274302601814, 0.023336296901106834, 0.030049478635191917, -0.03905851021409035, -0.007512369658797979, 0.013651588931679726, 0.03034009411931038, -0.03728576749563217, -0.013956733047962189, 0.031240995973348618, -0.010883491486310959, -0.054228559136390686, 0.016419686377048492, -0.0006638711784034967, 0.04240057244896889, 0.008747478947043419, -0.015068331733345985,

0.007840254344046116, 0.0177578404545784, 0.05462713539600372, -0.020746517926454544, ...]]

# Convert the embeddings to numpy array
embedding_matrix = np.array(embeddings)
embedding_matrix

array([[-0.01819243, -0.03411807, -0.02018314, ..., -0.00173733, -0.02522529, 0.00684396], [-0.01819243, -0.03411807, -0.02018314, ..., -0.00173733, -0.02522529, 0.00684396], [-0.00356826, -0.03058816, -0.01480166, ..., -0.00345601, -0.01368646, 0.02147833], ..., [-0.01836957, -0.03246572, -0.01109092, ..., 0.00375077, -0.00479223, 0.00559542], [-0.00718078, -0.02741507, -0.01103076, ..., 0.00263969, 0.00469953, -0.00361736], [-0.0362394 , -0.03605177, -0.01267173, ..., -0.00439255, -0.00796757, 0.00993099]])

# Verify the embedding matrix
print(f"Generated embeddings for {len(filtered_recipes)} recipes.")
print(f"Each embedding is of size {len(embeddings[0])}")

• Chuyển danh sách embeddings sang dạng numpy array gọi là embedding_matrix.

• Lý do dùng ma trận:

  • Dễ thao tác toán học (ví dụ: tìm công thức gần nhất, đo khoảng cách cosine, v.v.).

  • Tối ưu cho hiệu năng xử lý sau này.

Generated embeddings for 114 recipes.

Each embedding is of size 3072

IV. Retrieval System

Retrieval System là hệ thống cho phép bạn tìm kiếm thông minh trong một tập hợp tài liệu (ở đây là các công thức nấu ăn), không dựa vào từ khóa, mà dựa vào ngữ nghĩa của văn bản.

Ví dụ:

• Khi bạn tìm "cách làm bánh mì", hệ thống có thể trả về "Công thức bánh mì nướng giòn" mặc dù không có từ khóa "cách làm" — vì nó hiểu ngữ nghĩa nhờ vào embedding.

# Install the faiss-cpu library
!pip install faiss-cpu

• FAISS là thư viện mạnh mẽ từ Facebook dùng để tìm kiếm vector gần nhất một cách cực kỳ nhanh chóng.

• faiss-cpu: dành cho máy không có GPU.

# Import the faiss library
import faiss

# Print the embedding matrix shape
print(f"Embedding matrix shape: {embedding_matrix.shape}")

Embedding matrix shape: (114, 3072)

# Initialize the FAISS index for similarity search
index = faiss.IndexFlatL2(embedding_matrix.shape[1]) # Create a FAISS index with L2 distance metric
index.add(embedding_matrix) # Add the embeddings to the index

• IndexFlatL2: tạo chỉ mục dùng khoảng cách L2 (Euclidean distance).

• index.add(...): thêm các vector công thức (embedding_matrix) vào chỉ mục.

# Save the FAISS index to a file
faiss.write_index(index, "filtered_recipe_index.index")

Lưu chỉ mục FAISS vào file .index để tái sử dụng.

# Save the metadata for each recipe
metadata = [{'recipe_info': recipe['recipe_info'], # Include recipe information
             'image_path': recipe['image_path']} for recipe in filtered_recipes] # Include image path

# Write metadata to a JSON file with indentation
with open("recipe_metadata.json", "w") as json_file:
  json.dump(metadata, json_file, indent = 4)

Tạo danh sách metadata chứa thông tin gốc của từng công thức.

##Đây Generatelà thecâu embeddingstruy forvấn theđầu queryvào người dùng nhập.
query = "How to make bread?"
k = 5 # Number of top results to retrieve

#Dùng OpenAI để tạo embedding của truy vấn.
query_embedding = client.embeddings.create(
    input = [query],
    model = "text-embedding-3-large"
).data[0].embedding
print(f"The query embedding is {query_embedding}\n")

#Đưa embedding truy vấn về dạng vector 2 chiều để FAISS hiểu.
query_vector = np.array(query_embedding).reshape(1, -1)  # Convert embedding to a 2D numpy array for FAISS
print(f"The query vector is {query_vector}\n")

##search(...) Searchsẽ thetìm FAISSk indexkết forquả thegần nearestnhất neighborstrong vector space.
#Trả về:
#indices: chỉ số các công thức gần nhất.
#distances: khoảng cách tương ứng (càng nhỏ càng gần).
distances, indices = index.search(query_vector, min(k, len(metadata))) # Perform the search
print(f"The distances are {distances}\n")
print(f"The indices are {indices}\n")

# Store the indices and distances
stored_indices = indices[0].tolist()
stored_distances = distances[0].tolist()
print(f"The stored indices are {stored_indices}\n")
print(f"The stored distances are {stored_distances}\n")

# Print the metadata content for the top results
print("The metadata content is")
for i, dist in zip(stored_indices, stored_distances):
  if 0 <=i < len(metadata):
    print(f"Distance: {dist}, Metadata: {metadata[i]['recipe_info']}")

##Ghép Returntừng thechỉ resultssố kết quả (i) với thông tin gốc trong metadata và khoảng cách (dist) → kết quả trả về.
results = [(metadata[i]['recipe_info'], dist) for i, dist in zip(stored_indices, stored_distances) if 0 <= i < len(metadata)]
results # Output the results as a list of tuples containing recipe info and distance

The query embedding is [-0.019708624109625816, -0.028040051460266113, -0.022090725600719452, 0.016627300530672073, -0.04790274053812027, -0.048874542117118835, 0.03797139599919319, 0.01790723390877247, 0.0015688088024035096, 0.004139048047363758, -0.004817531909793615, -0.013901513069868088, -0.012917859479784966, -0.0015480691799893975, 0.03806620463728905, -0.02385655976831913, 0.016994688659906387, -0.016651002690196037, -0.007258888799697161, 0.002297660568729043, -0.03185615316033363, 0.02015897072851658, 0.014991827309131622, -0.007780343759804964, 0.006287086755037308, 0.017836127430200577, 0.003407233627513051, -0.011205354705452919, -0.03963056951761246, 0.042308952659368515, -0.0036798121873289347, 0.023690642789006233, -0.03465304523706436, -0.021154476329684258, -0.01591622456908226, 0.011329792439937592, -0.01473109982907772, 0.005253065377473831, 0.008515121415257454, 0.019412342458963394, -0.012408255599439144, 0.007792194839566946, 0.00038775798748247325, -0.008254393935203552, -0.012467511929571629, 0.016544342041015625, 0.06229015439748764, 0.019684921950101852, -0.015548836439847946, -0.011513486504554749, 0.046053946018218994, 0.008461790159344673, 0.03894319757819176, 0.00098513497505337, -0.02713935635983944, 0.06679362803697586, 0.012230487540364265, -0.04579322040081024, 0.001270305598154664, -0.01080241147428751, 0.018938293680548668, -0.01425705011934042, -0.026665305718779564, 0.029533307999372482, 0.014138538390398026, -0.0255275871604681, -0.003241316182538867, 0.0135341240093112, 0.0068796491250395775, 0.02782672829926014, 0.011513486504554749, -0.006310788914561272, 0.003179097082465887, 0.026357173919677734, 0.006595219019800425, 0.0067137316800653934, 0.002268032403662801, -0.04877973347902298, 0.030931755900382996, 0.04214303568005562, -0.02564609982073307, -0.0104409484192729, -0.006364119704812765, -0.01648508571088314, -0.02126113697886467, 0.040910504758358, 0.006915202829986811, -0.0465754009783268, 0.011098693124949932, -0.027115654200315475, 0.07020679116249084, -0.010215775109827518, 0.033302005380392075, 0.002308030379936099, 0.038374338299036026, 0.003054658882319927, 0.02602533809840679, -0.009149163030087948, 0.005090110469609499, -0.014458521269261837, -0.05589048191905022, -0.0019732327200472355, 0.002779117552563548, -0.01473109982907772, -0.025551289319992065, 0.044252555817365646, 0.027566000819206238, 0.03346792235970497, -0.015501431189477444, -0.09044872224330902, 0.008935840800404549, 0.009765427559614182, 0.006227830424904823, 0.0180494487285614, -0.019021252170205116, -0.011637925170361996, 0.005264916457235813, -0.00997874978929758, 0.03804250434041023, -0.0028872601687908173, 0.019281979650259018, 0.0195071529597044, 0.02411728724837303, 0.037923991680145264, -0.026926033198833466, -0.004444217775017023, -0.006577442400157452, -0.02014712058007717, 0.0014132612850517035, 0.01823906973004341, -0.006855946499854326, -0.004459031857550144, -0.03237760812044144, 0.029154067859053612, -0.002940590726211667, -0.018191665410995483, 0.023062527179718018, 0.013249694369733334, 0.0001649915793677792, 0.004076829180121422, 0.002882815897464752, 0.014197793789207935, -4.596987855620682e-05, -0.01013281662017107, 0.0011532744392752647, 0.02770821563899517, -0.006956682074815035, -0.015086637809872627, -0.006992235779762268, 0.030410300940275192, -0.050818149000406265, 0.0016043626237660646, -0.00036090752109885216, -0.019791582599282265, -0.022718841210007668, -0.010826114565134048, -0.006583367940038443, 0.0008688445668667555, 0.030505109578371048, 0.036359626799821854, 0.002708009909838438, -0.029628118500113487, 0.01755169779062271, 0.019530855119228363, -0.031619127839803696, 0.018559053540229797, -0.03133469820022583, 0.005380466114729643, -0.05133960396051407, 0.014517777599394321, -0.037923991680145264, 0.010832039639353752, 0.000179527880391106, 0.005107887554913759, -0.020976707339286804, 0.018950143828988075, 0.005842664744704962, -0.029533307999372482, 0.029628118500113487, 0.0005307136452756822, -0.026475686579942703, -0.017148755490779877, -0.009682469069957733, 0.03178504481911659, -0.006227830424904823, -0.01484961248934269, -0.014908868819475174, 0.04342297092080116, 0.004953821189701557, -0.005721189547330141, -0.008977320045232773, -0.008521046489477158, -0.03389456868171692, 0.02588312327861786, 0.027305273339152336, -0.036620352417230606, -0.03600408881902695, 0.005777482874691486, -0.02763710916042328, -0.008758071810007095, -0.01020984910428524, -0.011869024485349655, -0.010138741694390774, -0.017622804269194603, 0.004461994394659996, -0.008598079904913902, 0.025717206299304962, -0.024034328758716583, 0.03285165876150131, -0.014695546589791775, 0.02770821563899517, -0.010606866329908371, -0.012070495635271072, -0.015122191049158573, 0.006796690169721842, -0.006944830995053053, -0.03977278620004654, 0.010606866329908371, -0.015596241690218449, 0.013522272929549217, 0.004930119030177593, -0.009777278639376163, -0.02457948587834835, -0.02251737006008625, -0.017800573259592056, -0.0059582144021987915, 0.004257560707628727, 0.008236616849899292, 0.017077647149562836, -0.01416224054992199, -0.006642624270170927, -0.0001281416043639183, -0.005371577572077513, 0.012811197899281979, -0.011270536109805107, -0.012858603149652481, -0.009747650474309921, 0.014020025730133057, -0.022955866530537605, 0.017717614769935608, 0.018120557069778442, 0.03633592277765274, -0.05916142463684082, -0.02117817848920822, -0.016129547730088234, -0.0060915411449968815, -0.020478954538702965, -0.030813243240118027, 0.008124030195176601, -0.020550062879920006, -0.010992031544446945, 0.018618309870362282, -0.0001242529251612723, 0.0032176135573536158, 0.007987740449607372, -0.012692686170339584, 0.02645198442041874, 0.019222723320126534, -0.0009325449937023222, 0.06878463923931122, 0.007525541819632053, -0.03263833373785019, -0.02495872601866722, -0.024840213358402252, 0.006672251969575882, -0.03894319757819176, 0.013439314439892769, -0.0464094839990139, 0.02141520380973816, 0.022683287039399147, 0.044679202139377594, -0.04429996386170387, -0.038303229957818985, -0.009759502485394478, 0.018843483179807663, -0.02408173494040966, 0.015276257880032063, 0.008254393935203552, 0.0314532108604908, 0.049443405121564865, -0.0015243666712194681, 0.00889436062425375, 0.012858603149652481, 0.04991745203733444, 0.03714180737733841, 0.0026991216000169516, -0.04095790907740593, -0.0025435739662498236, -0.01020984910428524, -0.01858275569975376, -0.008218839764595032, -0.009966898709535599, -0.013214140199124813, 0.009889866225421429, -0.018760524690151215, 0.044181451201438904, 0.037236619740724564, -0.01663915067911148, 0.03534042090177536, 0.07997221499681473, -0.017646506428718567, -0.05939844995737076, -0.02145075798034668, -0.01085574273020029, -0.029841439798474312, 0.020905600860714912, -0.01576215960085392, -0.019495300948619843, -0.013723744079470634, 0.0025243156123906374, 0.02588312327861786, -0.03804250434041023, 0.011714957654476166, -0.03033919259905815, -0.03806620463728905, 0.021355947479605675, 0.004304965492337942, 0.02896444872021675, -0.002610237104818225, 0.0051641808822751045, -0.029367391020059586, -0.04515325278043747, -0.02901185303926468, 0.030694730579853058, 0.011637925170361996, 0.006156723015010357, 0.012906008400022984, -0.004556804429739714, 0.026665305718779564, 0.012266040779650211, 0.006121169310063124, -0.04958561807870865, -0.007164078764617443, -0.005881181452423334, 0.013557827100157738, -0.041692689061164856, 0.01675766333937645, 0.02931998483836651, -0.0255275871604681, -0.019436044618487358, 0.015821415930986404, -0.012751941569149494, -0.011157949455082417, -0.044631797820329666, 0.017231713980436325, 0.04598283767700195, 0.017350226640701294, 0.03590928018093109, -0.006482632365077734, -0.013901513069868088, 0.0004762719909194857, -0.0030487333424389362, 0.03268573805689812, 0.006210053339600563, -0.00914323702454567, -0.03531671687960625, -0.009729874320328236, -0.0014821465592831373, 0.016165101900696754, 0.022304046899080276, 0.04354148358106613, -0.021936658769845963, 0.01820351555943489, -0.004450143314898014, -0.0195071529597044, -0.02156927064061165, -0.028917042538523674, -0.0615316741168499, -0.0017880569212138653, -0.02946219965815544, -0.021095219999551773, 0.030007358640432358, 0.05565345659852028, 0.01454148069024086, 0.01721986196935177, -0.0016932470025494695, -0.022813651710748672, -0.008639559149742126, -0.036620352417230606, -0.0137948514893651, 0.021237434819340706, 0.022268492728471756, -0.0027924501337110996, -0.03633592277765274, -0.0048471600748598576, -0.03595668449997902, -0.0059404377825558186, 0.02182999812066555, 0.0032116880174726248, 0.025480180978775024, -0.03014957346022129, 0.007478137034922838, -0.03394197300076485, 0.022114427760243416, -0.002454689471051097, -0.02431875839829445, 0.04873232915997505, -0.03569595515727997, 0.03325460106134415, 0.025859421119093895, 0.03531671687960625, -0.03595668449997902, 0.0019510115962475538, -0.026878628879785538, -0.0032087252475321293, -0.00912545993924141, 0.02702084369957447, -0.05911402031779289, -0.029154067859053612, -0.036122601479291916, 0.02136779949069023, 0.020467104390263557, 0.023714344948530197, -0.039109114557504654, -0.016082141548395157, 0.028419291600584984, -0.02740008383989334, 0.033610135316848755, -0.024792809039354324, -7.425546937156469e-05, 0.009516551159322262, -0.004773089662194252, 0.013166735880076885, -0.014268901199102402, -0.033491626381874084, -0.002214701846241951, 0.01797834224998951, 0.018310176208615303, -0.030291788280010223, -0.005937474779784679, 0.0014977013925090432, -0.014458521269261837, 0.021498162299394608, 0.052192892879247665, 0.025290561839938164, -0.018618309870362282, -0.004222007002681494, 0.02870372124016285, -0.009510626085102558, 0.0076677571050822735, 0.038611363619565964, -0.011981611140072346, -0.0032442789524793625, 0.016496935859322548, 0.00820698868483305, -0.04614875838160515, -0.015619943849742413, -0.01782427541911602, 0.003505006432533264, -0.03247241675853729, -0.012917859479784966, -0.014470373280346394, 0.0022280344273895025, 0.035624850541353226, 0.01322599221020937, -0.01995749957859516, -0.03531671687960625, 0.05541643127799034, 0.02011156640946865, 0.008218839764595032, -0.015975480899214745, 0.012668983079493046, 0.0637597069144249, 0.011981611140072346, -0.014873315580189228, 0.019400492310523987, -0.035293012857437134, 0.003262055804952979, -0.018570903688669205, -0.004109419882297516, 0.014434819109737873, -0.002359879668802023, -0.03102656453847885, 0.012917859479784966, 0.014695546589791775, 0.00892991479486227, -0.0017243564361706376, 0.020893748849630356, -0.03913281857967377, -0.043754804879426956, -0.023797305300831795, 0.04095790907740593, -0.005694524385035038, -0.013166735880076885, 0.01767021045088768, -0.011821619234979153, -0.009439518675208092, 0.008058847859501839, 0.005392317660152912, 0.03875357657670975, 0.007928484119474888, 0.022043319419026375, 0.016402125358581543, 0.0027243054937571287, 0.028751125559210777, -0.011039436794817448, 0.0005581196746788919, 0.03659665212035179, 0.022801799699664116, -0.0013762260787189007, -0.022624030709266663, 0.014304455369710922, -0.03951205685734749, -0.0012273447355255485, 0.040554966777563095, -0.0038783205673098564, -0.012740090489387512, 0.02014712058007717, -0.01454148069024086, 0.056885987520217896, 0.04121863842010498, 0.0018399060936644673, 0.016093993559479713, -0.004568655975162983, -0.0072114840149879456, 0.0067670620046556, -0.0452006570994854, 0.022410709410905838, 0.020099714398384094, -0.005078259389847517, 0.0004933081800118089, 0.02789783664047718, 0.011602370999753475, -0.019779732450842857, -0.013427463360130787, -0.010357989929616451, 0.035932980477809906, 0.011092767119407654, 0.029912548139691353, -0.01945974864065647, 0.009386187419295311, 0.00368277495726943, 0.015252554789185524, 0.002872446086257696, 0.016212506219744682, -0.011045362800359726, -0.017125051468610764, -0.0012940081069245934, -0.00860400591045618, -0.020787088200449944, -0.03891949728131294, -0.026120148599147797, -0.004642725922167301, -0.004100531339645386, -0.005401205737143755, -0.0032027994748204947, 0.005709338467568159, -0.023074379190802574, 0.009587658569216728, -0.0030783615075051785, -0.00037053663982078433, -0.019898243248462677, -0.030054762959480286, 0.012514917179942131, 0.008295873180031776, 0.004613097757101059, 0.023868411779403687, -0.0034368617925792933, -0.005184920504689217, -0.0008073662174865603, 0.02206702157855034, -0.009054352529346943, 0.00763220340013504, -0.025717206299304962, -0.0021302616223692894, -0.0003312793851364404, -0.007335922215133905, 0.033728647977113724, 0.021355947479605675, 0.013439314439892769, -0.0007303330930881202, -0.006287086755037308, 0.03470045328140259, 0.015560687519609928, -0.0066604008898139, 0.02576461061835289, 0.006826318334788084, 0.013273396529257298, -0.010470576584339142, -0.0018917553825303912, 0.005256027914583683, 0.02901185303926468, -0.0491589717566967, -0.005250102374702692, -0.005605639889836311, 0.026238661259412766, 0.01721986196935177, -0.0019643441773951054, -0.013925215229392052, 0.02366694062948227, -0.00250061322003603, 0.006577442400157452, -0.018156111240386963, 0.015856968238949776, 0.020680425688624382, 0.013486719690263271, -0.023809155449271202, 0.037236619740724564, 0.00023943222186062485, -0.008035145699977875, 0.014268901199102402, -0.006962607614696026, -0.00523825129494071, 0.006903351284563541, 0.01904495432972908, -0.0059463633224368095, 0.024212097749114037, 0.012586024589836597, -0.0007792194955982268, 0.013723744079470634, -0.03344421833753586, 0.012443809770047665, -0.044939927756786346, -0.01653249002993107, -0.04126604273915291, -0.0266416035592556, -0.006156723015010357, -0.021355947479605675, 0.0016325092874467373, 0.004873825237154961, 0.027874132618308067, 0.0049804868176579475, 0.028253372758626938, 0.0014117797836661339, 0.025006132200360298, 0.014624439179897308, -0.023524725809693336, 0.014790356159210205, -0.005386391654610634, -0.02545647881925106, 0.019566409289836884, 0.006162648554891348, 0.030742134898900986, -0.01513404306024313, 0.0025820904411375523, 0.0018680528737604618, 0.013344503939151764, 0.03434491530060768, -0.01698283664882183, -0.020751534029841423, -0.013474867679178715, -0.008361054584383965, -0.014446670189499855, 0.004355333279818296, -0.023809155449271202, 0.005321209784597158, -0.006731508299708366, 0.02194850891828537, -0.007418880704790354, -0.005484164692461491, -0.017421333119273186, 0.005996731109917164, -0.003718328895047307, 0.021972212940454483, -0.011726808734238148, -0.03297017142176628, -0.0035079692024737597, 0.04714426025748253, -0.022837353870272636, 0.03401308134198189, -0.023524725809693336, 0.0032353904098272324, 0.028040051460266113, 0.031476911157369614, 0.004550878889858723, 0.011839396320283413, -0.00916101410984993, 0.026570497080683708, 0.006029321812093258, -0.021628526970744133, 0.02007601223886013, 0.005306396167725325, -0.009800981730222702, 0.03531671687960625, 0.012846752069890499, -0.014375562779605389, -0.014624439179897308, 0.02721046283841133, 0.049253784120082855, -0.028537802398204803, -0.009315080009400845, 0.031192483380436897, -0.005184920504689217, 0.011466081254184246, 0.006447078660130501, -0.009611361660063267, -0.03470045328140259, -0.004139048047363758, 0.0017021354287862778, -0.014577033929526806, -0.009552105329930782, 0.013297099620103836, 0.02839558757841587, 0.011637925170361996, -0.021628526970744133, -0.011709032580256462, -0.004616060759872198, 0.0269023310393095, 0.002054709941148758, 0.020253781229257584, -0.020360441878437996, 0.019068656489253044, -0.008426236920058727, 0.007318145129829645, 0.01473109982907772, 0.03083694539964199, 0.005827850662171841, 0.005857478827238083, -0.0450347401201725, -0.030410300940275192, 0.02431875839829445, 0.02977033331990242, -0.0060974666848778725, -0.035435229539871216, -0.004180527292191982, -0.004811606369912624, -0.0009480997687205672, -0.00226210686378181, -0.03102656453847885, 0.00368277495726943, -0.040863100439310074, -0.03256722912192345, -0.00582192512229085, -0.007247037719935179, 0.014920719899237156, 0.04427625983953476, 0.038421742618083954, 0.019104210659861565, -0.03384716063737869, -0.018215367570519447, 0.001903606578707695, -0.0017021354287862778, -0.02289661020040512, 0.00535380095243454, 0.024413568899035454, 0.016935432329773903, -0.007946261204779148, 0.023145485669374466, -0.01595177873969078, 0.03839804232120514, 0.013676338829100132, -0.00534491240978241, -0.014908868819475174, 0.014280753210186958, -0.0054752761498093605, -0.009889866225421429, -0.00403534946963191, 0.021320393308997154, -0.02877482771873474, -0.0020473028998821974, -0.00950470007956028, -0.0023361770436167717, 0.0017421332886442542, 0.01782427541911602, 0.02324029617011547, 0.021201880648732185, -0.004959746729582548, 0.010790560394525528, 0.020016755908727646, -0.020064162090420723, -0.010067634284496307, -0.0014754802687093616, 0.0009510625968687236, -0.006393747869879007, -0.012046792544424534, -0.026807520538568497, -0.0039020231924951077, 0.020775236189365387, 0.023299552500247955, 0.024840213358402252, 0.025551289319992065, -0.011306090280413628, 0.008485493250191212, -0.0018650899874046445, -0.028656315058469772, -0.011780139990150928, 0.011673478409647942, -0.025029834359884262, 0.013593380339443684, 0.04088680446147919, 0.0005262694321572781, 0.012858603149652481, 0.01509848888963461, -0.004189415834844112, -0.009842460975050926, 0.02450837939977646, 0.004106457345187664, -0.00831957533955574, 0.010067634284496307, -0.014802208170294762, -0.022315898910164833, 0.01644953154027462, -0.012586024589836597, 0.02327585034072399, 0.03289906308054924, 0.012538619339466095, -0.015868820250034332, -0.024532081559300423, 0.008846956305205822, 0.017350226640701294, 0.017812425270676613, -0.02564609982073307, 0.004829383455216885, 0.022434411570429802, -0.007300368510186672, 0.016058439388871193, -0.0004973820177838206, 0.006921128369867802, -0.017184307798743248, 0.009569882415235043, 0.04529546573758125, 0.040934208780527115, -0.006518186070024967, -0.017646506428718567, 0.041431959718465805, -0.0029435534961521626, 0.03206947445869446, -0.0035612997598946095, -0.0008925470756366849, 0.023453617468476295, -0.011537189595401287, -0.003193911164999008, -0.023714344948530197, 0.02958071231842041, -0.007685533724725246, 0.004130159504711628, 0.004944933112710714, -0.01276379358023405, 0.011673478409647942, -0.00896546896547079, 0.009012873284518719, -0.010523907840251923, 0.005096036475151777, -0.018428688868880272, -0.01911606267094612, 0.022078873589634895, 0.02385655976831913, 0.006293012294918299, 0.02514834702014923, 0.004464957397431135, -0.021936658769845963, -0.05503719300031662, 0.04766571521759033, -0.0033213121350854635, -0.020846344530582428, 0.016745813190937042, -0.030671028420329094, 0.009836534969508648, -0.006221904885023832, -0.04240376129746437, -0.0003059103328268975, 0.001555476221255958, 0.015335514210164547, 0.03413159027695656, 0.037805479019880295, 0.008651410229504108, -0.01106906495988369, 0.009623212739825249, 0.032235391438007355, -0.009777278639376163, 0.01823906973004341, -0.008574377745389938, -0.005039742682129145, 0.004861974157392979, -0.00847364217042923, -0.0780286118388176, 0.005893032532185316, -0.002768747741356492, 0.0046753170900046825, -0.0057982224971055984, 0.009356559254229069, 0.0031198407523334026, -0.02225664258003235, 0.01858275569975376, -0.01188680063933134, -0.0007484803209081292, -0.04963302239775658, 0.028988150879740715, -0.017089499160647392, 0.019436044618487358, -0.0054574995301663876, -0.003543522907420993, -0.03408418595790863, 0.009261749684810638, -0.004799755290150642, 0.0021702595986425877, -0.0061033922247588634, 0.007158153224736452, 0.029177770018577576, -0.0033746426925063133, -0.003899060422554612, 0.01579771190881729, -0.013143032789230347, -0.0018562015611678362, 0.03026808612048626, 0.006210053339600563, 0.012965264730155468, -0.006844095420092344, 0.002220627386122942, -0.011282387189567089, -0.0208818968385458, 0.03206947445869446, -0.026997141540050507, 0.03481896594166756, 0.034178998321294785, -0.012106048874557018, -0.00013369688531383872, -0.018464243039488792, -0.018452391028404236, 0.006731508299708366, -0.007602575235068798, 0.021166328340768814, 0.030694730579853058, 0.007946261204779148, 0.008444014005362988, -0.015359216369688511, 0.007187781389802694, -0.025409074500203133, -0.006618921644985676, 0.0270682480186224, 0.003193911164999008, 0.012242338620126247, 0.019601963460445404, -0.006980384700000286, 0.0038723950274288654, 0.01732652261853218, -0.006968533154577017, 0.007697385270148516, 0.02801634930074215, 0.0319746658205986, 0.01701839081943035, 0.02469799853861332, 0.015975480899214745, -0.004850123077630997, -0.024058032780885696, 0.012716388329863548, 0.013771149329841137, -0.012111974880099297, -0.011732734739780426, -0.02476910687983036, -0.004127196967601776, -0.0035553742200136185, -0.03206947445869446, 0.003505006432533264, 0.014968125149607658, 0.002078412566334009, 0.027115654200315475, 0.016354721039533615, 0.003232427639886737, -0.003979056142270565, -0.018677566200494766, 0.004147936590015888, -0.011531263589859009, -0.005048631224781275, -0.03389456868171692, -0.013937066309154034, -0.016959134489297867, 0.0126334298402071, -0.011845321394503117, -0.0001836943265516311, 0.03401308134198189, -0.019530855119228363, 0.02164037711918354, 0.00917879119515419, 0.01381855458021164, -0.02863261289894581, 0.0020858196076005697, 0.007454434409737587, 0.018274623900651932, 0.004085717257112265, 0.022813651710748672, -0.00430200295522809, 0.011620148085057735, -0.006903351284563541, 0.010743155144155025, 0.018073152750730515, -0.009155088104307652, 0.024366164579987526, -0.014932571910321712, 0.046930938959121704, -0.03014957346022129, -0.018618309870362282, 0.007193706929683685, -0.03375235199928284, 0.010512055829167366, 0.007199632469564676, 0.003087249817326665, 0.037615858018398285, 0.027352677658200264, -0.007069269195199013, -0.00978913065046072, 0.02740008383989334, -0.012799346819519997, 0.006719657219946384, -0.013652636669576168, 0.020941153168678284, -0.019720476120710373, 0.023714344948530197, -0.012538619339466095, -0.009718022309243679, -0.045864325016736984, -0.0025761649012565613, -0.034605640918016434, 0.011424602009356022, 0.005430833902209997, 0.025124644860625267, -0.006778913550078869, -0.047428689897060394, -0.004953821189701557, 0.011537189595401287, -0.009101757779717445, 0.016473233699798584, -0.003288721200078726, -0.011839396320283413, -0.027044545859098434, 0.014091133140027523, -0.0026072743348777294, 0.010897221975028515, -0.0039020231924951077, 0.050012264400720596, 0.02965182065963745, 0.010061709210276604, 0.023524725809693336, -0.0002549869823269546, -0.0052915820851922035, 0.017527993768453598, 0.019400492310523987, -0.00940989051014185, -0.0023272887337952852, -0.04328075423836708, -0.002577646169811487, 0.016082141548395157, -0.024840213358402252, -0.007448508869856596, 0.005045668687671423, -0.01497997622936964, 0.0062693096697330475, -0.05110257863998413, 0.007412955164909363, -0.004358296282589436, 0.004977523814886808, 0.019791582599282265, -0.009415815584361553, 0.03033919259905815, -0.009949121624231339, 0.0009658766211941838, -0.012171231210231781, -0.027874132618308067, -0.006612996105104685, -0.009587658569216728, 0.028182266280055046, -0.012870454229414463, 0.017042092978954315, -0.044821418821811676, 0.04117123410105705, 0.00386943225748837, 0.010636494494974613, -0.03835063427686691, 0.002390989102423191, 0.006950756534934044, -0.021083367988467216, 0.0035346345975995064, 0.008829179219901562, -0.013474867679178715, 0.017658358439803123, 0.014221496880054474, 0.003688700729981065, 0.004355333279818296, -0.03014957346022129, 0.012372702360153198, 0.001675470033660531, 0.010672047734260559, 0.0016428790986537933, -0.013640785589814186, -0.004352370742708445, 0.022588476538658142, -0.04254597797989845, -0.011922354809939861, 0.04000981152057648, -0.020277483388781548, 0.0005714523140341043, 0.03190355747938156, 0.028940746560692787, 0.0013725225580856204, -0.014174091629683971, -0.02995995245873928, 0.024674296379089355, 0.024295056238770485, -0.013249694369733334, -0.0065537397749722, -0.00689742574468255, 0.010144667699933052, 0.017871681600809097, 0.0027361568063497543, -0.02133224532008171, -0.026617901399731636, 0.013747447170317173, 0.047428689897060394, -0.003460564184933901, -0.012017164379358292, -0.016840621829032898, -0.006476706825196743, -0.021770741790533066, 0.009338783100247383, 0.015536985360085964, 0.0026946773286908865, -0.010221701115369797, 0.010920924134552479, -0.009315080009400845, -0.0020087864249944687, 0.005460462067276239, 0.0008310687262564898, -0.007762567140161991, 0.026807520538568497, 0.020313037559390068, 0.0019050879636779428, 0.01007948536425829, 0.009006948210299015, -0.003359828609973192, -0.01532366219907999, -0.002349509857594967, -0.0037064775824546814, 0.030291788280010223, 0.026807520538568497, 0.01139497384428978, 0.0010962403612211347, -0.027968943119049072, -0.012514917179942131, 0.009445443749427795, 0.001260676421225071, 0.019945649430155754, 0.004630874842405319, 0.024188395589590073, 0.008805477060377598, -0.03275684639811516, -0.025219453498721123, 0.03019697777926922, -0.012550470419228077, 0.01767021045088768, 0.03273314610123634, -0.019519004970788956, -0.003904985962435603, 0.011697180569171906, 0.01141275092959404, -0.005981917027384043, -0.003345014527440071, -0.06366489827632904, 0.0022472927812486887, -0.019910095259547234, 0.008017368614673615, -0.009374336339533329, -0.010375767014920712, -0.0017288007074967027, 0.003984981682151556, -0.02072783187031746, 0.0012880824506282806, 0.013202289119362831, 0.01603473722934723, 0.0016354721738025546, 0.014055578969419003, 0.0020917451474815607, -0.010749081149697304, -0.0328279547393322, 0.003164282999932766, -0.017279118299484253, 0.006304863374680281, 0.016082141548395157, 0.029794035479426384, -0.03984389454126358, -0.0004777534049935639, -0.012858603149652481, 0.02965182065963745, -0.03453453257679939, 0.02676011621952057, -0.005176032427698374, -0.018369432538747787, 0.01701839081943035, -0.022529220208525658, 0.025669801980257034, 0.0027583776973187923, -0.0016532490262761712, 0.0033746426925063133, 0.0006755210924893618, -0.017006540670990944, 0.00955803133547306, 0.015584389679133892, 0.01785982958972454, -0.025859421119093895, -0.008580302819609642, 0.006992235779762268, 0.0037005520425736904, 0.02339436113834381, -0.039180222898721695, -0.010067634284496307, 0.017682060599327087, 0.0006821874412707984, -0.028111157938838005, -0.004547916352748871, 0.02160482294857502, -0.01425705011934042, -0.01729097031056881, -0.006571516394615173, 0.022955866530537605, 0.013783000409603119, 0.002028044778853655, 0.006215979345142841, 0.00014452966570388526, 0.007436657790094614, 0.014067430049180984, 0.013012669049203396, 0.01526440680027008, 0.015679199248552322, 0.03256722912192345, -0.015110339969396591, -0.009872089140117168, 0.018618309870362282, 0.003884246340021491, -0.00914323702454567, 0.011240907944738865, -0.016200654208660126, -0.014020025730133057, 0.006358194164931774, -0.012574173510074615, 0.025788314640522003, -0.004053126554936171, -0.022588476538658142, 0.003505006432533264, 0.0190805085003376, 0.020905600860714912, 0.012550470419228077, 0.0007329255458898842, 0.0006166351959109306, 0.018369432538747787, 0.01179199106991291, 0.01404372788965702, 0.03550633788108826, 0.020052310079336166, 0.0016517675248906016, -0.0031465061474591494, -0.006370045244693756, -0.024603189900517464, -0.002716898452490568, 0.018037598580121994, 0.013628934510052204, 0.040436454117298126, -0.009747650474309921, 0.015027381479740143, 0.009202493354678154, -0.022114427760243416, 0.005392317660152912, -0.009872089140117168, -0.019862690940499306, -0.03676256909966469, 0.0010451318230479956, 0.039369843900203705, 0.003982019145041704, -0.017954640090465546, 0.0033124235924333334, 0.007010012865066528, 0.01057131215929985, -0.014932571910321712, 0.004823457449674606, -0.010506130754947662, -0.003647221252322197, 0.018559053540229797, 0.014612588100135326, 0.0007977370987646282, -0.009131385944783688, 0.011365345679223537, -0.020514508709311485, 0.015560687519609928, 0.016236208379268646, -0.03297017142176628, 0.011673478409647942, 0.006618921644985676, 0.013605231419205666, -0.011572742834687233, 0.015335514210164547, -0.010695750825107098, -0.004130159504711628, -0.04128974676132202, 0.0009214344900101423, -0.021083367988467216, -0.023903965950012207, -0.024366164579987526, -0.01526440680027008, -0.013735595159232616, 0.02808745577931404, -0.0013184512499719858, 0.003899060422554612, -0.0066070700995624065, -0.011726808734238148, -0.017243564128875732, 0.0012051237281411886, -0.0014488149899989367, -0.009208419360220432, -0.014707397669553757, 0.004909378942102194, -0.016864323988556862, -0.021901104599237442, -0.0019510115962475538, -0.018760524690151215, -0.0266416035592556, -0.002069524023681879, 0.003199836704879999, -0.010547609999775887, 0.0030931755900382996, -0.019898243248462677, 0.02137964963912964, 0.0041716392152011395, 0.03221169114112854, -0.006073764059692621, 0.014802208170294762, 0.003777584992349148, -0.0027880058623850346, 0.04633837565779686, 0.009919493459165096, 0.007898855954408646, -0.009018799290060997, 0.010073560290038586, 0.0006640402134507895, -0.010766858235001564, -0.01072537899017334, -0.023951370269060135, -0.005007151979953051, -0.009315080009400845, 0.0043819984421133995, -0.025551289319992065, 0.009800981730222702, 0.032496120780706406, -0.004755313042551279, -0.0046486519277095795, -0.030908051878213882, -0.01509848888963461, 0.015738455578684807, 0.026617901399731636, -0.018760524690151215, 0.0003066510253120214, -0.00403534946963191, -0.02289661020040512, 0.004429403692483902, 0.020099714398384094, -9.143052011495456e-05, -0.018227217718958855, -0.0041864532977342606, -0.005703412927687168, -0.0097535764798522, 0.023939520120620728, -0.007329996209591627, -0.016046589240431786, -0.020170822739601135, -0.010411320254206657, -0.012799346819519997, 0.005371577572077513, -0.01463629025965929, -0.004758275579661131, -0.01911606267094612, -0.012206784449517727, -0.010162444785237312, -0.012609726749360561, -0.025693504139780998, 0.027803026139736176, -0.0068796491250395775, -0.009095832705497742, 0.011857172474265099, 0.009860238060355186, 0.020775236189365387, 0.003256130265071988, -0.016733961179852486, -0.00883510522544384, 0.016093993559479713, -0.0013599306112155318, 0.022908460348844528, -0.01633101888000965, -0.0007666275487281382, -0.036170005798339844, -0.004070903640240431, 0.0037686966825276613, -0.042877811938524246, -0.02225664258003235, -0.02255292423069477, -0.01119942869991064, 0.0059582144021987915, -0.01756354793906212, -0.029912548139691353, -2.7382862754166126e-05, -0.018867185339331627, 0.017089499160647392, 0.001551031949929893, 0.04821087419986725, -0.0211426243185997, 0.017338374629616737, 0.00393165135756135, 0.023346956819295883, 0.013439314439892769, 0.02416469343006611, -0.0035761138424277306, 0.024792809039354324, 0.007715161889791489, 0.001363634131848812, 0.008852881379425526, -0.002955404808744788, 0.018559053540229797, 0.02225664258003235, 0.02172333560883999, -0.008805477060377598, 0.0028872601687908173, 0.012372702360153198, 0.0018976809224113822, 0.008823253214359283, 0.000278689491096884, -0.001063649426214397, -0.012503066100180149, 0.02671271190047264, 0.007282591424882412, -0.0014377044280990958, -0.02901185303926468, -0.027305273339152336, -0.00029850329156033695, -0.04716796427965164, -0.0008962505962699652, 0.007175930310040712, -0.03289906308054924, -0.018879037350416183, 0.01828647404909134, 0.020123418420553207, 0.01660359650850296, 0.02342991530895233, 0.0060915411449968815, 0.006073764059692621, 0.009534328244626522, -0.02004045993089676, -0.0064293015748262405, -0.015963630750775337, 0.007134450599551201, 0.006684103514999151, -0.006064875982701778, -0.0033035350497812033, 0.008035145699977875, 0.022031469270586967, 7.55517030484043e-05, 0.010008377954363823, 0.014707397669553757, 0.0030205866787582636, 0.006618921644985676, -0.00016258429968729615, 0.014742951840162277, 0.012858603149652481, -0.0008488455787301064, 0.01961381360888481, -0.016627300530672073, 0.009326932020485401, 0.003670923877507448, 0.017077647149562836, 0.02373804897069931, -0.0212255846709013, 0.0001074945175787434, -0.009860238060355186, 0.007412955164909363, -0.02335880883038044, 0.029557010158896446, -0.020870046690106392, -0.005214548669755459, -0.01995749957859516, 0.025812016800045967, 0.016959134489297867, 0.009806906804442406, -0.019056806340813637, -0.024840213358402252, 0.01701839081943035, 0.014944422990083694, 0.00464568892493844, 0.012917859479784966, 0.003253167262300849, 0.0051701064221560955, 0.03465304523706436, -0.00594340031966567, -0.017468739300966263, 0.03408418595790863, 0.016212506219744682, -0.028182266280055046, -0.011916428804397583, -0.008029219694435596, 0.0036679611075669527, -0.007792194839566946, -0.0019332347437739372, 0.00404720101505518, 0.02030118741095066, -0.0171013493090868, -0.0012473437236621976, 0.002989477012306452, 0.01319043803960085, -0.01820351555943489, 0.011649776250123978, 0.00404720101505518, 0.010695750825107098, 0.0016428790986537933, 0.003232427639886737, 0.002390989102423191, 0.001991009572520852, 0.017765019088983536, -0.0131074795499444, -0.0035227832850068808, -0.029983654618263245, -0.027566000819206238, -0.016248060390353203, 0.0003610926796682179, 0.02324029617011547, -0.025859421119093895, -0.01394891832023859, 0.0390143059194088, -0.014920719899237156, -0.019673069939017296, 0.021687783300876617, -0.00858622882515192, 0.006014508195221424, -0.013261545449495316, 0.006672251969575882, -0.01416224054992199, 0.023524725809693336, 0.006814467255026102, -0.011146098375320435, 0.007892930880188942, 0.0028902229387313128, 0.02126113697886467, -0.0014873315813019872, 0.006512260530143976, 0.01141275092959404, -0.00736554991453886, 8.008851000340655e-05, -0.04664650931954384, -0.005217511672526598, 0.02633347176015377, -0.0034250104799866676, 0.005673784762620926, 0.009172865189611912, 0.02495872601866722, 0.004461994394659996, -0.003884246340021491, -0.020846344530582428, -0.02167593128979206, 0.003072435734793544, -0.010138741694390774, -0.0020991521887481213, 0.014458521269261837, -0.010873518884181976, 0.014174091629683971, -0.012058643624186516, 0.011869024485349655, 0.0066070700995624065, 0.01767021045088768, 0.0019199020462110639, 0.03178504481911659, -0.0010088373674079776, -0.013865958899259567, -0.004506436642259359, 0.027163058519363403, -0.006453004200011492, -0.04083939641714096, 0.0016813956899568439, -0.011276462115347385, 0.02488761954009533, -0.020775236189365387, -0.007649980019778013, -0.012206784449517727, -0.0021658153273165226, 0.005498978774994612, -0.01036391593515873, -0.003733142977580428, -0.0015762158436700702, -0.014008174650371075, -0.0255275871604681, -0.03209317848086357, -0.013463016599416733, -0.013972620479762554, 0.0076736826449632645, -0.01425705011934042, 0.0009451369405724108, 0.01036391593515873, -0.007104822900146246, 0.013344503939151764, 0.01438741385936737, 0.08926359564065933, -0.005558235105127096, -0.015584389679133892, 0.0020621169824153185, -0.011383122764527798, -0.006772988010197878, -0.008325501345098019, -0.0024087659548968077, -0.003469452727586031, -0.007092971354722977, 0.0043819984421133995, -0.010861667804419994, -0.0014769616536796093, -0.005090110469609499, -0.004174601752310991, 0.02359583228826523, -0.024437271058559418, 0.017196159809827805, -0.029414795339107513, -0.007762567140161991, -0.006109317764639854, 0.00044257001718506217, 0.00631671492010355, -0.00011943835124839097, 0.016117695719003677, 0.006595219019800425, 0.006932979449629784, -0.0035464856773614883, 0.015359216369688511, 0.015975480899214745, 0.005596751347184181, -0.026167554780840874, -0.005223437212407589, 0.017421333119273186, 0.002044340129941702, -7.532023300882429e-05, 0.0012214191956445575, 0.005356763955205679, 0.005341949872672558, 0.0008088476024568081, -0.008331426419317722, -0.009190642274916172, 0.006239681504666805, -0.01775316894054413, 0.018025746569037437, 0.015939926728606224, 0.011116470210254192, 0.012834900990128517, -0.014174091629683971, 0.005181957967579365, 0.015584389679133892, -0.009172865189611912, 0.004755313042551279, -0.015584389679133892, -0.009309154935181141, -0.0171013493090868, 0.02614385075867176, 0.010529832914471626, 0.018606457859277725, -0.0008606968331150711, 0.0073951780796051025, -0.003688700729981065, -0.004023498389869928, 0.020277483388781548, 0.018179813399910927, 0.017883531749248505, -0.03346792235970497, 0.00028535581077449024, -0.002220627386122942, -0.008325501345098019, -0.008260319009423256, -0.02382100746035576, -0.007519616279751062, 0.0023939518723636866, 0.017053944990038872, 0.02652309089899063, -0.008752145804464817, 0.01595177873969078, 0.01345116551965475, 0.008100327104330063, -0.018961995840072632, -0.0008658817387185991, -0.014920719899237156, 0.005327135790139437, 0.003155394457280636, -0.007946261204779148, -0.030362894758582115, 0.013368207029998302, -0.005727115087211132, 0.015288108959794044, -0.00511381309479475, -0.005869330372661352, -0.013178586959838867, -0.004583470057696104, 0.03294646739959717, 0.007756641134619713, 0.022375155240297318, -0.012372702360153198, 0.003976093605160713, 0.0183931365609169, -0.009712097235023975, -0.01927012763917446, -0.001359189860522747, -0.0018458317499607801, -0.014600737020373344, -0.010482428595423698, -0.002506538759917021, 0.0019836025312542915, 0.01188680063933134, -0.009006948210299015, -0.0262149590998888, -0.02908296138048172, -0.0006277457578107715, 0.028348183259367943, 0.01013281662017107, -0.01526440680027008, -0.016698407009243965, -0.00014527037274092436, 0.0031524316873401403, 0.005984880030155182, -0.0053152842447161674, -0.007987740449607372, -0.010636494494974613, -0.004467920400202274, -0.014174091629683971, -0.013652636669576168, 0.004272374790161848, -0.007727012969553471, -0.002256181091070175, -0.004121271427720785, -0.0013391909888014197, -0.009652840904891491, 0.008900286629796028, -0.025622395798563957, -0.001187346875667572, 0.014221496880054474, 0.01636657305061817, -0.011229056864976883, 0.012070495635271072, 0.026973439380526543, -0.018997550010681152, -0.03171393647789955, 0.010992031544446945, 0.006838169414550066, -0.0259542316198349, 0.0005958955152891576, 0.01085574273020029, -0.009469146840274334, -0.00997874978929758, 0.002457652473822236, 0.008396608754992485, -0.011981611140072346, 0.014375562779605389, 0.007187781389802694, 0.0010140223894268274, 0.02495872601866722, 0.0028561505023390055, 0.010109113529324532, -0.019637515768408775, -0.003587965155020356, 0.01679321750998497, -0.006026359274983406, -0.022541072219610214, -0.009990601800382137, -0.002250255551189184, -0.014245199039578438, 0.0228610560297966, 0.0032502044923603535, -0.01473109982907772, 0.0011192021192982793, 0.011655701324343681, 0.016473233699798584, -0.0008562526199966669, 0.0050041889771819115, 0.010411320254206657, 0.0073892525397241116, 0.022434411570429802, -0.003552411450073123, 0.013628934510052204, -0.020704129710793495, -0.003060584655031562, -0.0016295465175062418, 0.015821415930986404, -0.0045271762646734715, 0.004467920400202274, -0.015821415930986404, -0.008657336235046387, 0.0026946773286908865, -0.010766858235001564, -0.018606457859277725, 0.032614633440971375, 0.0019421231700107455, -0.00048108657938428223, -0.0032294648699462414, 0.012574173510074615, -0.024674296379089355, -0.003033919259905815, -0.02015897072851658, 0.014280753210186958, -0.00039294292218983173, 0.007324070669710636, 0.014612588100135326, 0.029533307999372482, -0.014434819109737873, 0.006026359274983406, 0.025598693639039993, -0.0090247243642807, 0.009113608859479427, -0.037426237016916275, 0.0018087966600432992, -0.01854720152914524, 0.007158153224736452, 0.004432366229593754, -0.0007103341049514711, -0.010186146944761276, 0.005261953920125961, -0.017042092978954315, 0.005765631794929504, -0.015513282269239426, -0.00488567678257823, -0.0005303433281369507, -0.00997874978929758, 0.011709032580256462, 0.0008984726737253368, 0.00285318773239851, 0.01878422684967518, -0.012906008400022984, -0.024140991270542145, -0.00950470007956028, -0.00403534946963191, -0.01564364694058895, 0.01093277521431446, 0.025740908458828926, 0.025930529460310936, 0.005288619082421064, -0.007401104085147381, -0.012266040779650211, 0.04019942879676819, -0.0017406519036740065, -0.01015059370547533, 0.010654271580278873, 0.0010480947094038129, -0.018333880230784416, -0.004488660022616386, -0.009741725400090218, 0.01002022996544838, 0.02137964963912964, 0.0027257867623120546, 0.00748998811468482, -2.6179219275945798e-05, -0.005007151979953051, -0.0052915820851922035, 0.003632407169789076, -0.009338783100247383, 0.012704537250101566, 0.013783000409603119, -0.015240703709423542, -0.005167143885046244, -0.00028442993061617017, 0.033230897039175034, -0.004882713779807091, 0.022221088409423828, -0.004023498389869928, -0.002693195827305317, 0.015169596299529076, 0.015975480899214745, -0.0036798121873289347, 0.00439088698476553, 0.004518288187682629, -0.0252668596804142, 0.0013110442087054253, -0.0022961790673434734, -0.00831957533955574, -0.014600737020373344, -0.0025820904411375523, -0.026665305718779564, -0.014956274069845676, 0.017314672470092773, 0.020502658560872078, -0.011021659709513187, 0.017622804269194603, 0.0022221088875085115, -0.004379035905003548, 0.0008369943243451416, -0.023832857608795166, 0.008313650265336037, 0.013771149329841137, 0.00034072334528900683, -0.007300368510186672, 0.017397630959749222, 0.0259542316198349, 0.024152841418981552, 0.004414589609950781, -0.00394053990021348, -0.015928076580166817, 0.0013547457056120038, -0.03827952966094017, 0.008331426419317722, -0.005276768002659082, -0.0006203387165442109, -0.00794033519923687, -0.004085717257112265, 0.01131794136017561, 0.011466081254184246, 0.015003679320216179, 0.013522272929549217, 0.009321006014943123, -0.01038169302046299, 0.01544217485934496, -0.00267690047621727, -0.0007366290665231645, -9.962454350898042e-05, 0.010387618094682693, -0.016579894348978996, -0.0035405601374804974, 0.011335717514157295, -0.012100123800337315, -0.019210871309041977, 9.45785068324767e-05, -0.020739682018756866, 0.00990171730518341, 0.025930529460310936, -0.011466081254184246, -0.0011651257518678904, -0.012894157320261002, -0.00984838604927063, 0.012106048874557018, -0.019068656489253044, 0.03538782522082329, -0.020123418420553207, 0.005519718397408724, -0.01671025902032852, 0.026428280398249626, -0.009492848999798298, -0.002245811279863119, 0.012965264730155468, 0.012621577829122543, -0.013486719690263271, -0.037497345358133316, -0.00488567678257823, -0.01797834224998951, -0.01889088749885559, 0.021912956610322, 0.008236616849899292, 0.010606866329908371, -0.006346343085169792, 0.008740294724702835, 0.0014021507231518626, -0.012396404519677162, -0.0005714523140341043, -0.01070760190486908, -0.006524111609905958, 0.01416224054992199, -0.013273396529257298, -0.010026155039668083, -0.017658358439803123, 0.019068656489253044, 0.00463680038228631, -0.03095545805990696, 0.01497997622936964, 0.003973130602389574, 0.012953413650393486, -0.004467920400202274, 0.009475071914494038, 0.0020250817760825157, 0.024484677240252495, 0.01007948536425829, -0.0048204949125647545, -0.029912548139691353, 0.030694730579853058, -0.010008377954363823, -0.03891949728131294, 0.0022635881323367357, 0.04083939641714096, 0.007804046384990215, -0.003827952779829502, -0.006932979449629784, -0.012431958690285683, 0.001992490841075778, 0.00894769188016653, -0.03228279575705528, -0.003579076612368226, -0.0006333010387606919, 0.023572130128741264, 0.021118922159075737, -0.007857376709580421, 0.028680019080638885, 0.004621986299753189, 0.020123418420553207, -0.010565387085080147, 0.012562322430312634, -0.03752104938030243, 0.010482428595423698, -0.010014303959906101, -0.005721189547330141, 0.007875153794884682, -0.0027524521574378014, 0.005851553287357092, -0.011750511825084686, 0.008752145804464817, -0.024484677240252495, 0.005226400215178728, 5.6617482186993584e-05, 0.015536985360085964, 0.02225664258003235, 0.006636698264628649, -0.018179813399910927, -0.01756354793906212, 0.0060974666848778725, -0.0007492210133932531, -0.002828003838658333, 0.0054871272295713425, 0.002910962561145425, 0.003241316182538867, 0.007288516964763403, -0.012775644659996033, 0.00214655720628798, -0.0025924602523446083, 0.021782591938972473, 0.0030487333424389362, 0.01794278807938099, 0.02392766810953617, 0.005525643937289715, 0.0038575809448957443, 0.0052915820851922035, 0.007809971924871206, -0.014055578969419003, -0.004464957397431135, 0.005584900267422199, -0.002225071657449007, -0.013273396529257298, -0.008230690844357014, 0.007075194735080004, -0.0025954232551157475, 0.006932979449629784, 0.0012080864980816841, 0.02965182065963745, -0.008195137605071068, -0.010180220939218998, -0.011821619234979153, -0.018262771889567375, -0.000443681055912748, 0.016271762549877167, -0.0048264204524457455, -0.016390275210142136, 0.003822027239948511, -0.0027820803225040436, 0.006737433839589357, -0.006127094849944115, -0.0201945248991251, 0.021047815680503845, 0.021841848269104958, 0.01945974864065647, -0.012277891859412193, 0.016662852838635445, -0.011691255494952202, -0.015288108959794044, -0.01070760190486908, -0.0024072846863418818, 0.004746424499899149, 0.01061279233545065, 0.011578668840229511, 0.00035387082607485354, 0.014802208170294762, 0.029983654618263245, -0.0062633841298520565, 0.00955803133547306, 0.023121783509850502, 0.008088476024568081, -0.004882713779807091, -0.004236821085214615, 0.0076796081848442554, -0.02289661020040512, -0.006530037149786949, -0.0005777482874691486, -0.009765427559614182, -0.01020984910428524, -0.0014273346168920398, -0.014150389470160007, -0.0015258480561897159, 0.001737689133733511, -0.010310584679245949, -0.006731508299708366, 0.0045271762646734715, 0.021628526970744133, 0.01667470484972, 0.057312630116939545, -0.00032072438625618815, -0.009232121519744396, -0.02671271190047264, -8.476419316139072e-05, 0.008289947174489498, 0.005190846510231495, 0.0173620767891407, 0.003946465440094471, -0.007608500774949789, -0.043114837259054184, -0.04370740056037903, 0.0014621475711464882, -0.0039020231924951077, -0.008444014005362988, -0.01416224054992199, 0.009155088104307652, -0.0033568658400326967, 0.01441111695021391, -0.00042923734872601926, -0.009581733494997025, -0.004355333279818296, -0.012301594950258732, 0.009759502485394478, 0.00018674973398447037, 0.007525541819632053, -0.02870372124016285, -0.009919493459165096, -0.0011806804686784744, -0.008361054584383965, -0.0019124950049445033, 0.0009888384956866503, -0.0024902434088289738, 0.01767021045088768, -0.012740090489387512, -0.004936044570058584, 0.003976093605160713, -0.016354721039533615, -0.013522272929549217, 0.008823253214359283, -0.02820596843957901, -0.002048784401267767, -0.004696056712418795, -0.020526360720396042, 0.00963506381958723, 0.018085002899169922, 0.001857682946138084, 0.03064732439815998, -0.007904781959950924, -0.022908460348844528, 0.039488356560468674, -0.008289947174489498, -0.009261749684810638, -0.0038338785525411367, -0.017966490238904953, -0.005534532479941845, 0.01847609505057335, -0.013332652859389782, -0.014577033929526806, -0.014766653999686241, -0.008562525734305382, 0.006405598949640989, 0.0008081069099716842, -0.0036916634999215603, -0.0020102676935493946, 0.012751941569149494, 0.004248672164976597, -0.010109113529324532, -0.021118922159075737, -0.007537393365055323, -0.00011249425733694807, 0.011495709419250488, -0.004500511102378368, -0.007436657790094614, -0.012574173510074615, 0.0013265990419313312, -0.02813486009836197, -0.0017050981987267733, -0.002245811279863119, 0.00988394021987915, -0.0024902434088289738, -0.005013077519834042, 0.004746424499899149, -0.0003755363868549466, -0.00942766759544611, 0.019068656489253044, -0.013237843289971352, 0.002593941753730178, -0.013463016599416733, 0.01579771190881729, -0.006790764629840851, -0.006719657219946384, -0.01565549708902836, -0.016117695719003677, -0.0075018396601080894, -0.0016888027312234044, -0.012906008400022984, -0.009872089140117168, 0.00914323702454567, 0.018831631168723106, -0.013652636669576168, -0.014648141339421272, 0.0018680528737604618, 0.007484062574803829, 0.007821823470294476, -0.017468739300966263, -0.009670617990195751, -0.006186351180076599, -0.0046753170900046825, -0.0059196981601417065, 0.020585617050528526, -0.0031761343125253916, 0.03157172352075577, -0.019720476120710373, -0.017113201320171356, 0.008046996779739857, 0.012811197899281979, 0.024034328758716583, 0.009226195514202118, -0.016627300530672073, -0.006690029054880142, 0.012491215020418167, -0.01966121979057789, -0.01610584557056427, -0.005967102944850922, -0.012467511929571629, -0.023038825020194054, -0.011892726644873619, -0.002989477012306452, 0.0019317532423883677, -0.01319043803960085, -0.025432776659727097, 0.01602288708090782, -0.02392766810953617, 0.009759502485394478, -0.011750511825084686, -0.010357989929616451, 0.03273314610123634, 0.014790356159210205, -0.02777932398021221, -0.00027609703829512, 0.013119330629706383, 0.005433796904981136, 0.004642725922167301, -0.018428688868880272, 0.01992194727063179, 0.00010036525054601952, -0.000557378982193768, -0.039488356560468674, -0.019222723320126534, -0.026428280398249626, 0.0027406008448451757, 0.0002501724229659885, 0.004379035905003548, 0.016662852838635445, -0.008064773865044117, 0.003013179637491703, -0.01057131215929985, 0.016769515350461006, -0.0017406519036740065, -0.004467920400202274, 0.003164282999932766, -0.0048530856147408485, 0.013854107819497585, -0.004787903744727373, -0.008432161994278431, 0.0007381105097010732, -0.011400899849832058, -0.024840213358402252, 0.0013865958899259567, -0.01522885262966156, -0.01618880406022072, -0.010340212844312191, -0.005241214297711849, 0.020241931080818176, 0.0008740294724702835, 0.013854107819497585, 0.0053300983272492886, -0.000338316080160439, -0.00523825129494071, -0.03512709587812424, -0.005718227010220289, -0.01532366219907999, -0.009469146840274334, -0.007353698834776878, -0.0017302820924669504, 0.012052718549966812, 0.000768108933698386, -0.04072088375687599, -0.015062935650348663, -0.03195096179842949, -0.011501635424792767, 0.00128067540936172, -0.000839957152493298, -0.01238455343991518, 0.004838271532207727, 0.002005823655053973, 0.03114507719874382, -6.284401479206281e-06, 0.02827707678079605, -0.010340212844312191, 0.002389507833868265, 0.020893748849630356, 0.002811708487570286, -0.00039183185435831547, -0.0032739071175456047, -0.0017317634774371982, -0.015892522409558296, -0.001151793054305017, -0.005528606940060854, -0.021190030500292778, 0.0015614018775522709, 0.013048223219811916, -0.0018265735125169158, -0.006387822329998016, -0.006903351284563541, 0.002214701846241951, 0.004340519197285175, 0.0001497145858593285, 0.014138538390398026, 0.018606457859277725, -0.0025198713410645723, -0.0029998470563441515, -0.005910809617489576, -0.021427055820822716, 0.011359420605003834, -0.0059463633224368095, 0.013510421849787235, 0.007288516964763403, 0.013297099620103836, -0.015276257880032063, 0.010322436690330505, -0.041431959718465805, 0.0016043626237660646, -0.021901104599237442, 0.005134552717208862, -0.011104618199169636, 0.01912791281938553, -0.0041923788376152515, -0.004174601752310991, 0.017990192398428917, -0.0017984267324209213, 0.002506538759917021, 0.019838986918330193, 0.017587250098586082, 0.014304455369710922, -0.018961995840072632, 0.023192889988422394, -0.009795055724680424, -0.003419084707275033, -0.0045093996450304985, -0.003724254434928298, 0.009605435654520988, -0.006808541715145111, 0.017919085919857025, -0.003579076612368226, 0.003324274905025959, -0.014446670189499855, 0.014553331770002842, -0.017089499160647392, -0.0004555323102977127, -0.0114305280148983, 0.008444014005362988, 0.012988966889679432, 0.021818146109580994, -0.005208623129874468, 0.011560891754925251, -0.03707070276141167, 0.011258685030043125, -0.01272823940962553, 0.012799346819519997, -0.01663915067911148, 0.020170822739601135, 0.00386943225748837, -0.021747037768363953, 0.005730078089982271, -0.02133224532008171, 0.00858622882515192, 0.003081324277445674, -0.008212914690375328, -0.01820351555943489, 0.012218635529279709, 0.00476123858243227, -0.009113608859479427, 0.0033005722798407078, 0.0030694729648530483, 0.01763465628027916, 0.006002656649798155, -0.007833674550056458, -0.045745816081762314, 0.01729097031056881, 0.004598284140229225, -0.014197793789207935, 0.004589395597577095, -0.0062693096697330475, 0.003724254434928298, -0.0035287088248878717, -0.007472211495041847, 0.007424806244671345, 0.04102901741862297, -0.01794278807938099, 0.011768288910388947, 0.004438292235136032, 0.015074786730110645, -0.011359420605003834, 0.019945649430155754, 0.012645280919969082, -0.0009006948093883693, -0.016496935859322548, -0.03178504481911659, -0.015157745219767094, -0.003650184255093336, 0.014126686379313469, -0.006956682074815035, -0.010280956514179707, -0.020704129710793495, -0.018642012029886246, 0.017302820459008217, 0.0024295055773109198, -0.0049804868176579475, -0.015051083639264107, 0.007590723689645529, 0.014695546589791775, -0.001551031949929893, -0.03342051804065704, -0.002059154212474823, 0.001265120692551136, -0.008076624944806099, 0.003496117889881134, -0.017693912610411644, 0.012787495739758015, 0.01629546470940113, -0.007804046384990215, 0.012123825959861279, 0.007685533724725246, 0.007964038290083408, -0.014233347959816456, 0.0034990806598216295, -0.0011962353019043803, -0.004328668117523193, 0.008449939079582691, 0.008829179219901562, -0.0013740039430558681, 0.003419084707275033, -0.007596649695187807, -0.020289335399866104, 0.006079689599573612, 0.014837761409580708, -0.020502658560872078, 0.0035464856773614883, 0.01394891832023859, -0.008876584470272064, 0.0016265836311504245, -0.014209645800292492, -0.0208818968385458, -0.01820351555943489, 0.002112484769895673, 0.011009808629751205, 0.022149981930851936, -0.013202289119362831, 0.0040442380122840405, 0.010109113529324532, -0.03375235199928284, -0.025977933779358864, 0.005872292909771204, 0.0023732122499495745, -0.00712852505967021, -0.00016323241288773715, -0.016200654208660126, -0.015501431189477444, -0.0018858297262340784, -0.0035612997598946095, -0.006180425640195608, -0.0020724867936223745, -0.012645280919969082, -0.013996322639286518, -0.007175930310040712, -0.020905600860714912, 0.03325460106134415, -0.004781978204846382, -0.022588476538658142, 0.009528403170406818, 0.005656007677316666, -0.017729464918375015, -0.01307192537933588, 0.009546179324388504, -0.01234899926930666, -0.019910095259547234, 0.00476123858243227, 0.024792809039354324, -0.00013258583203423768, -0.01770576275885105, 0.023370658978819847, 0.020135268568992615, -0.001986565301194787, -0.007288516964763403, -0.010808337479829788, 0.010399469174444675, -0.01912791281938553, -0.004678279627114534, 0.003996833227574825, 0.019258277490735054, 0.0012458623386919498, 0.012070495635271072, -0.004358296282589436, -0.00028683722484856844, -0.007300368510186672, 0.009854312054812908, 0.009931345470249653, -0.0008451420580968261, -0.008846956305205822, -0.007116673979908228, -0.016236208379268646, -0.010903147049248219, 0.0032768698874861, -0.01945974864065647, 0.0051523298025131226, -0.014399264939129353, 0.03458194062113762, 0.009492848999798298, 0.07404658943414688, 0.010304659605026245, 0.0029850329738110304, -0.02030118741095066, 0.011863098479807377, -0.006035247817635536, 0.02671271190047264, -0.009599510580301285, 0.00015175151929724962, 0.008989171124994755, 0.010559461079537868, -0.00368277495726943, 0.007033715024590492, -0.0012821567943319678, -0.01166162732988596, -0.03982019051909447, -0.01904495432972908, 0.012094197794795036, 0.0029642931185662746, 0.026878628879785538, 0.011489784345030785, -0.0036116675473749638, -0.003703514812514186, -0.002038414590060711, -0.03370494768023491, -0.0007584798149764538, -0.011590519919991493, -0.01801389642059803, -0.013273396529257298, -0.014197793789207935, -0.02671271190047264, 0.023252146318554878, -0.010091337375342846, 0.004070903640240431, 0.02488761954009533, 0.010138741694390774, -0.013048223219811916, -0.005276768002659082, -0.016544342041015625, 0.003350940067321062, -0.02015897072851658, 0.01509848888963461, -0.00020221190061420202, 0.007608500774949789, -0.011353494599461555, 0.006802615709602833, 0.01748058944940567, -0.014340009540319443, 0.008141807280480862, -0.012811197899281979, -0.022090725600719452, -0.016082141548395157, 0.010275031439960003, 0.004547916352748871, -3.3887161407619715e-05, 0.019187169149518013, 0.008769922889769077, 0.012337148189544678, 0.01454148069024086, -1.1075823749706615e-05, 0.014209645800292492, -0.017279118299484253, -0.01969677209854126, 0.00047701271250844, -0.012420106679201126, 0.017397630959749222, 0.017053944990038872, 0.021557418629527092, 0.0024902434088289738, 0.017042092978954315, -0.031429506838321686, 0.006962607614696026, 0.02007601223886013, 0.011335717514157295, -0.010672047734260559, -0.01335635595023632, -0.002216183114796877, -0.0010066153481602669, 0.007910707965493202, 0.020929303020238876, -0.0060915411449968815, 0.0005296026356518269, -0.00631671492010355, 0.02229219675064087, 0.02232774905860424, -0.006583367940038443, 0.004438292235136032, 0.01202309038490057, 0.017421333119273186, -0.015785861760377884, 0.00462791183963418, -0.003359828609973192, 0.016425829380750656, -0.00014378895866684616, 0.03256722912192345, -0.005365652032196522, 0.007620351854711771, 0.028727423399686813, 0.020170822739601135, 0.015785861760377884, -0.020455252379179, -0.033373113721609116, 0.025906827300786972, 0.012645280919969082, 0.018037598580121994, -0.003919799812138081, -0.011074990965425968, 0.018404986709356308, 0.00807069893926382, 0.007655905559659004, 0.007004087325185537, -0.006684103514999151, 0.002530241385102272, 0.013439314439892769, 0.014897017739713192, 0.017184307798743248, -0.0015599203761667013, -0.005427871365100145, 0.005407131742686033, 0.015928076580166817, 0.015394770540297031, 0.022280344739556313, -0.005436759442090988, -0.0060856156051158905, -0.0011621629819273949, -0.012372702360153198, -0.005617490969598293, 0.0018191664712503552, 0.000674039707519114, 0.023181039839982986, -0.021972212940454483, 0.0221736840903759, -0.008313650265336037, -0.009344708174467087, -0.009907642379403114, -0.001841387478634715, 0.003759808139875531, 0.016935432329773903, 0.007934410125017166, 0.004097568802535534, 0.00486789969727397, 0.01694728434085846, -0.004310891032218933, 0.012443809770047665, 0.007649980019778013, 0.0035257460549473763, 0.005235288292169571, -0.00012027164484607056, -0.01973232626914978, 0.011999388225376606, -0.008995096199214458, -0.005543421022593975, 0.004790866747498512, 0.00784552562981844, 0.013024521060287952, -0.011436454020440578, 0.010879444889724255, 0.009718022309243679, 0.004112382885068655, -0.007199632469564676, -0.004334593657404184, 0.015868820250034332, 0.000338316080160439, 0.01425705011934042, 0.009949121624231339, -0.011217205785214901, -0.00714630214497447, -0.01103351078927517, 0.009943196550011635, 0.00333908898755908, 0.001555476221255958, 0.012834900990128517, 0.018559053540229797, 0.012550470419228077, 0.022505518049001694, -0.009498775005340576, 0.0025954232551157475, -0.009249898605048656, 0.008201062679290771, 0.00044405143125914037, -0.007401104085147381, -0.008556600660085678, 0.003730179974809289, 0.012183082289993763, -0.02438986673951149, -0.028253372758626938, 0.015122191049158573, 0.00847364217042923, -0.0006647809059359133, 0.0010910554556176066, -0.006293012294918299, 0.008278096094727516, -0.017385778948664665, 0.004097568802535534, -0.017314672470092773, 0.00522936275228858, 0.003336125984787941, 0.008574377745389938, 0.009652840904891491, 0.014434819109737873, -0.013617083430290222, -0.009498775005340576, 0.005664896219968796, 0.013783000409603119, -0.002272476674988866, -0.01847609505057335, -0.0032294648699462414, 0.004841234534978867, 0.0038042503874748945, 0.013510421849787235, -0.0157266054302454, -0.012846752069890499, -0.005967102944850922, 0.00942766759544611, -0.010529832914471626, 0.024295056238770485, -0.03327830135822296, 0.018843483179807663, -0.03206947445869446, 0.01995749957859516, -0.014280753210186958, 0.005235288292169571, -0.0008081069099716842, 0.00879955105483532, 0.01547772902995348, -0.003226502100005746, 0.027992645278573036, -0.0018591644475236535, -0.0015317737124860287, -0.01854720152914524, -0.03308868408203125, 0.00750776519998908, -0.02034859172999859, -0.0006162648787721992, 0.005016040522605181, -0.029746631160378456, 0.0010110595030710101, 0.011560891754925251, 0.018724970519542694, 0.02335880883038044, -0.008212914690375328, 0.024982430040836334, -0.02232774905860424, -0.01473109982907772, 0.017504291608929634, -0.014458521269261837, -0.0005588603671640158, -0.0011814212193712592, 0.010239477269351482, -0.0017228750512003899, -0.008076624944806099, 0.013166735880076885, -2.6248661015415564e-05, 0.005048631224781275, -0.0013369688531383872, -0.0011117950780317187, -0.0040323869325220585, -0.007952187210321426, -0.006204127799719572, -0.01381855458021164, -0.012538619339466095, -0.01637842319905758, -0.004310891032218933, -0.0011436453787609935, -0.025124644860625267, -0.02763710916042328, -0.00856845173984766, -0.0038516554050147533, -0.00788107980042696, -0.005404168739914894, -0.002577646169811487, -0.017338374629616737, 0.011833470314741135, -0.010926850140094757, 0.00393165135756135, -0.006417450495064259, -0.013463016599416733, -4.469679333851673e-05, -0.01345116551965475, -0.005895995534956455, -0.02072783187031746, 0.003656109794974327, -0.02335880883038044, 0.001665100222453475, -0.003946465440094471, 0.02827707678079605, 0.015738455578684807, -0.007371475920081139, 0.010861667804419994, -0.013308950699865818, 0.033373113721609116, 0.019649367779493332, 0.01297711580991745, -0.016781365498900414, -0.010399469174444675, -0.0010821670293807983, 0.0057685947977006435, 0.013972620479762554, -0.0090247243642807, -0.009469146840274334, -0.016899878159165382, 0.0009762464906089008, -0.001654730411246419, -0.011116470210254192, 0.010630568489432335, -0.029414795339107513, 0.010215775109827518, -0.017693912610411644, -0.004310891032218933, -0.026191256940364838, 0.006358194164931774, 0.02427135407924652, 0.004447180312126875, 0.0067670620046556, -0.016236208379268646, 0.05527421832084656, 0.005241214297711849, 0.0004907157272100449, -0.00858622882515192, -0.022766245529055595, 0.0046693915501236916, 0.008556600660085678, -0.0029524420388042927, 0.005872292909771204, -0.013060074299573898, -0.02545647881925106, -0.034984882920980453, 0.016805067658424377, 0.012858603149652481, 0.0022132203448563814, -0.011602370999753475, -0.022967716678977013, -0.004539027810096741, -0.014209645800292492, 0.030315490439534187, 0.004944933112710714, -0.02813486009836197, 0.0070574176497757435, -0.012514917179942131, 0.007809971924871206, 0.023181039839982986, -0.002542092464864254, 0.00048145692562684417, -0.022588476538658142, 0.021083367988467216, 0.03247241675853729, 0.008147732354700565, -0.0259542316198349, -0.008331426419317722, -0.021581120789051056, -0.01588067039847374, 0.0037420312874019146, 0.02538537234067917, 0.030125871300697327, -0.00761442631483078, 0.014233347959816456, -0.0026013487949967384, 0.02076338604092598, 0.00835512951016426, 0.010357989929616451, -0.024342462420463562, -0.0058130365796387196, -0.031737640500068665, -0.0019510115962475538, 0.017231713980436325, -0.00415978766977787, 0.005084184929728508, 0.02416469343006611, -0.002708009909838438, 0.02160482294857502, -0.00788107980042696, 0.0180494487285614, -0.005528606940060854, -0.0062693096697330475, 0.005700449924916029, 0.030102167278528214, -0.008331426419317722, -0.0040649776346981525, 0.0012628985568881035, 0.008538823574781418, 0.022339601069688797, 0.0037272172048687935, 0.01579771190881729, 0.007934410125017166, 0.015868820250034332, -0.006932979449629784, 0.009060278534889221, 0.0020369330886751413, 0.00831957533955574, 0.016402125358581543, -0.011697180569171906, -0.021581120789051056, -0.012965264730155468, 0.008278096094727516, 0.00084588275058195, 0.007353698834776878, 0.00534491240978241, 0.0022265531588345766, 0.021272988989949226, 0.013474867679178715, -0.002722823992371559, 0.02934368886053562, 0.008331426419317722, -0.01618880406022072, 0.006630772724747658, -0.01706579513847828, -0.012586024589836597, -0.003543522907420993, 0.012562322430312634, 0.003676849417388439, 0.036241114139556885, 0.0072233350947499275, 0.015382918529212475, -0.00809440203011036, 0.0026428280398249626, -0.01070760190486908, 0.025906827300786972, 0.02813486009836197, 0.003256130265071988, -0.015892522409558296, 0.004613097757101059, -0.017682060599327087, -0.007418880704790354, -0.015122191049158573, -0.01725541613996029, 0.006761136464774609, 0.006287086755037308, 0.0011969759361818433, 0.027992645278573036, 0.02401062659919262, -0.0208818968385458, -0.0048264204524457455, -0.012846752069890499, -0.0016991725424304605, 0.022801799699664116, 0.013415612280368805, 0.009326932020485401, -0.002900592749938369, -0.015359216369688511, -0.011946056969463825, 0.013036372140049934, -0.012337148189544678, -0.00017869459406938404, 0.012526768259704113, -0.02392766810953617, -0.003336125984787941, 0.01667470484972, -0.022564774379134178, -0.003336125984787941, 0.007815897464752197, -0.016307316720485687, 0.0263808760792017, -0.01603473722934723, 0.009955047629773617, 0.023986924439668655, 0.004728647414594889, 0.007596649695187807, -0.010565387085080147, -0.00279096863232553, 0.005448610987514257, 0.005368615034967661, -0.01653249002993107, -0.030979160219430923, -0.01610584557056427, 0.0067670620046556, -0.00748998811468482, 0.032164283096790314, 3.877116978401318e-05, -0.018333880230784416, -0.037189215421676636, 0.019601963460445404, 0.011940131895244122, -0.004296076949685812, 0.009155088104307652, 0.023868411779403687, 0.011223130859434605, 0.007383326999843121, 0.01595177873969078, 0.0037805477622896433, 0.0003329459868837148, 0.020976707339286804, 0.021059665828943253, 0.01945974864065647, -0.01225418969988823, -0.005433796904981136, 0.012325297109782696, -0.005149366799741983, -0.007750715594738722, -0.004897527862340212, -0.007525541819632053, -0.013119330629706383, -0.010067634284496307, 0.0023643237072974443, -0.004470882937312126, 0.018191665410995483, -0.02614385075867176, 0.007638128940016031, -0.015525134280323982, 0.021557418629527092, 0.03650183975696564, -0.02026563324034214, -0.0016562117962166667, 0.013214140199124813, -0.015430323779582977, -0.0015347364824265242, -0.011495709419250488, -0.0010851297993212938, -0.005733040627092123, 0.009220270439982414, -0.02007601223886013, -0.023026973009109497, -0.0002814671315718442, -0.014008174650371075, -0.006340417079627514, -0.009629138745367527, 0.03600408881902695, 0.017492441460490227, 0.00782774854451418, -0.0010517982300370932, -0.0026220884174108505, 0.006133020389825106, 0.02274254336953163, 0.01068982481956482, 0.0005118257249705493, -0.0072292606346309185, -0.002900592749938369, 0.00940989051014185, -0.0030842870473861694, -0.008005517534911633, 0.00025202418328262866, -0.028229670599102974, 0.00822476577013731, -0.0011488302843645215, 0.005256027914583683, -0.010269105434417725, 0.0004944192478433251, -0.018985697999596596, -0.008562525734305382, -0.011892726644873619, 0.011406825855374336, 0.013711892999708652, -0.009611361660063267, 0.007999591529369354, 0.01179199106991291, 0.003371679922565818, -0.0007099637878127396, -0.021308543160557747, 0.019210871309041977, -0.014197793789207935, -0.003116877982392907, -0.01579771190881729, 0.009332857094705105, -0.0348900705575943, -0.011554965749382973, -0.0007310737855732441, -0.016591746360063553, 0.011460156179964542, 0.001967306947335601, 0.006073764059692621, -0.012621577829122543, 0.01809685491025448, -0.016058439388871193, 0.0064233760349452496, -0.009243972599506378, -0.015145894140005112, -0.017741316929459572, -0.004334593657404184, 0.008017368614673615, -0.00394053990021348, -0.0262149590998888, 0.017812425270676613, -0.018618309870362282,282, 0.014517777599394321, 0.0027317125350236893, -0.011122395284473896, -0.010849816724658012, -0.03183244913816452, -0.024342462420463562, 0.019945649430155754, 0.011975685134530067, 0.0007351476815529168, 0.0019021251937374473, 0.013143032789230347, 0.0057982224971055984] The query vector is [[-0.01970862 -0.02804005 -0.02209073 ... 0.00190213 0.01314303 0.00579822]] The distances are [[1.128458 1.1365714 1.1599339 1.160224 1.2071426]] The indices are [[17 9 2 14 10]] The stored indices are [17, 9, 2, 14, 10] The stored distances are [1.128458023071289, 1.1365714073181152, 1.1599339246749878, 1.1602239608764648, 1.2071425914764404] The metadata content is Distance: 1.128458023071289, Metadata: Here’s the structured information extracted from the recipe image: ### Recipe Title: Nut Bread and Oatmeal Bread ### Ingredients: #### Nut Bread: - 2½ Cups of Flour - 3 Teaspoons of Baking Powder - ¾ Cup of Milk - ½ Cup of Sugar

- Muffins - Traditional Recipes This structured format is suitable for embedding in a Retrieval Augmented Generation (RAG) system.

# Define a function to query the embeddings
def query_embeddings(query, index, metadata, k = 5):
  # Generate the embeddings for the query
  query_embedding = client.embeddings.create(
      input = [query],
      model = "text-embedding-3-large"
  ).data[0].embedding
  print(f"The query embedding is {query_embedding}\n")
  query_vector = np.array(query_embedding).reshape(1, -1)
  print(f"The query vector is {query_vector}\n")

  # Search faiss index
  distances, indices = index.search(query_vector, min(k, len(metadata)))
  # print(f"The distances are {distances}\n")
  # print(f"The indices are {indices}\n")

  # Store the indices and distances
  stored_indices = indices[0].tolist()
  stored_distances = distances[0].tolist()
  print(f"The stored indices are {stored_indices}\n")
  print(f"The stored distances are {stored_distances}\n")

  # # Print the metadata content
  # print("The metadata content is")
  # for i, dist in zip(stored_indices, stored_distances):
  #   if 0 <=i < len(metadata):
  #     print(f"Distance: {dist}, Metadata: {metadata[i]['recipe_info']}")

  # Return the results
  results = [(
      metadata[i]['recipe_info'], dist) for i, dist in zip(
          stored_indices, stored_distances) if 0 <= i < len(metadata)]
  return results

# Test the retrieval system
query = "chocolate query"
results = query_embeddings(query, index, metadata)
print(f"The results are {results}")

# Combine the results into a single string
def combined_retrived_content(results):
  combined_content = "\n\n".join([result[0] for result in results]) # Join the recipe information with double newlines
  return combined_content

# Get the combined content from results
combined_content = combined_retrived_content(results)
print(f"The combined content is {combined_content}")

V. Generative System

# Define the system prompt
system_prompt3 = f"""
You are highly experienced and expert chef specialized in providing cooking advice.
Your main task is to provide information precise and accurate on the combined content.
You answer diretly to the query using only information from the provided {combined_content}.
If you don't know the answer, just say that you don't know.
Your goal is to help the user and answer the {query}
"""

# Define function to retrieve a response from the API
def generate_response(query, combined_content, system_prompt):
  response = client.chat.completions.create(
      model = model,
      messages = [
          {"role": "system", "content": system_prompt3}, # Provide system prompt for guidance
          {"role": "user", "content": query}, # Provide the query as user input
          {"role": "assistant", "content": combined_content} # Provide the combined content from the results
      ],
      temperature = 0, # Set temperature to 0 for deterministic output
  )
  return response

# Get the results from the API
query = "How to make bread?"
combined_content = combined_retrived_content(results)
response = generate_response(query, combined_content, system_prompt3)

# Display the outcome
get_gpt_response()

I'm sorry, but the provided content does not include a recipe for making bread. If you have a specific bread recipe in mind or need guidance on a particular type of bread, please let me know!

# Get the results
query = "Get me the best chocolate cake recipe"
combined_content = combined_retrived_content(results)
response = generate_response(query, combined_content, system_prompt3)

# Display the outcome
get_gpt_response()

I'm sorry, but I don't have a specific chocolate cake recipe available. However, I can provide you with a chocolate sauce recipe if you're interested in making a sauce to accompany a cake. Would you like that?

VI. Rag system

# Build the function for Retrieval-Augmented Generation (RAG)
def rag_system(query, index, metadata, system_prompt, k = 5):
  # Retrieval System: Retrieve relevant results based on the query
  results = query_embeddings(query, index, metadata, k)

  # Content Merge: Combine the retrieved content into a single string
  combined_content = combined_retrived_content(results)

  # Generation: Generate a response based on the query and combined content
  response = generate_response(query, combined_content, system_prompt)

  # Return the generated response
  return response

# Test with a different query
query2 = "I want something vegan"
response = rag_system(query2, index, metadata, system_prompt3)
get_gpt_response()