(defun rcd-embeddings-create-for-llm-responses ()
"Create and store vector representations of text responses from a
language model that lack existing embeddings.
This function interacts with the user to process log entries in `log'
table where `l.log_logtypes = 8', indicating they are related to large
language models (LLMs). It checks for missing embeddings by performing
an SQL query on database connection specified as variable `rcd-db'. For
each entry without existing embedding, it retrieves text from the column
named `log_description'.
Non-nil means that a warning message is displayed if no embeddings can
be created. Return t if successful creation of new entries in the
'embeddings' table occurs.
Important arguments:
- The database connection variable `rcd-db' must point to an active and
valid SQL database.
The function uses helper functions such as `rcd-sql-list',
`rcd-db-get-entry', `rcd-llm-get-embedding', and others for
processing. It ensures that each log entry with a missing embedding is
processed, embeddings are generated using the LLM response text from
`log_description' column, inserted into database table 'embeddings'
along with their respective IDs.
If an error occurs during this process or if no valid embeddings can be
created by `rcd-llm-get-embedding', it issues warning
messages. Otherwise, a success message is displayed for each embedding
successfully stored in the database."
(interactive)
(let ((list (rcd-sql-list "SELECT l.log_id
FROM log l
LEFT JOIN embeddings e ON e.embeddings_referencedid = l.log_id
WHERE l.log_logtypes = 8
AND e.embeddings_referencedid IS NULL"
rcd-db)))
(while list
(let* ((id (pop list))
(input (rcd-db-get-entry "log" "log_description" id rcd-db))
(embeddings (rcd-llm-get-embedding input)))
(cond ((not embeddings) (rcd-warning-message "Could not create embeddings."))
(t (rcd-db-embeddings-insert id embeddings input 4)
(rcd-message "Created embedding for LLM response ID %s" id)))))))