[FRIAM] Gauging interest..

[Marcus Daniels]

I’m closing-in on a full download of Dataset 9 of the Epstein Transparency Act. (I have the rest.) I’m thinking of building a vector database (e.g. pgvector for Postgres). I was thinking of wrapping a MCP server around it so LLMs can get a directory of articles and then summarize, or cross-reference sets of them. RAG is what Perplexity does, but apparently, they don’t have the content yet. 

I imagine a SETI-at-home type project to reduce the data. Another analogy that comes to mind is annotations of the genome: Line all the documents up and then slowly fill in the summaries. The vector database could help inform how to combine documents for consumption within context window limits (PCA vicinity). 

I could keep my Max subscription on it and make some progress, but really such a project needs tens or hundreds of workers. 

Marcus 





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20260206/5de03c4c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 8422 bytes
Desc: not available
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20260206/5de03c4c/attachment.bin>