It's a bit technical, I haven't found any pre-packaged software to do what I'm doing yet.

First I installed https://github.com/openai/whisper , the speech-to-text model that OpenAI released back when they were less blinded by dollar signs. I wrote a Python script that used it to go through all of the audio files in the directory tree where I'm storing this stuff and produced a transcript that I stored in a .json file alongside it.

For the LLM, I installed https://github.com/LostRuins/koboldcpp/releases/ and used the https://huggingface.co/unsloth/Qwen3-30B-A3B-128K-GGUF model, which is just barely small enough to run smoothly on my RTX 4090. I wrote another Python script that methodically goes through those .json files that Whisper produced, takes the raw text of the transcript, and feeds it to the LLM with a couple of prompts explaining what the transcript is and what I'd like the LLM to do with it (write a summary, or write a bullet-point list of subject tags). Those get saved in the .json file too.

Most recently I've been experimenting with creating an index of the transcripts using those LLM results and the Whoosh library in Python, so that I can do local searches of the transcripts based on topics. I'm building towards writing up something where I can literally tell it "Tell me about Uncle Pete" and it'll first search for the relevant transcripts and then feed those into the LLM with a prompt to extract the relevant information from them.

If you don't find the idea of writing scripts for that sort of thing literally fun (like me) then you may need to wait a bit for someone more capable and more focused than I am to create a user-friendly application to do all this. In the meantime, though, hoard that data. Storage is cheap.

That's awesome! Thank you!

If you don’t find the idea of writing scripts for that sort of thing literally fun...

I absolutely do. What I find as a potential showstopper for me right now, is that I don't have a nonintegrated GPU, which makes complex LLMs hard to run. Basically, if I can't push the processing to CPU, I'm looking at around 2-5 seconds per token; it's rough. But I like your workflow a lot, and I'm going to try to get something similar going with my incredibly old hardware, and see if CPU-only processing of this would be something feasible (though, I'm not super hopeful there).

And, yes, I, too, am aware of the hallucinations and such that come from the technology. But, honestly, for this non-critical use case, I don't really care.

I only just recently discovered that my installation of Whisper was completely unaware that I had a GPU, and was running entirely on my CPU. So even if you can't get a good LLM running locally you might still be able to get everything turned into text transcripts for eventual future processing.

A long time ago, I had the idea for a startup to keep digital material, including accounts, passwords, old documents, etc. in a digital vault that would be released to the next-of-kin when someone dies. It would also convert documents to newer formats so your old unpublished WordPerfect novel could be opened and read by the grandkids (should they choose).

Problem is, nobody would (or should) trust a startup with that material. This is stuff that should be around for many decades and most startups go out of business.

This could be a non-profit funded by participants and government grants.

Bitwarden does all that. If you pay the subscription you get a GB of storage and delegate emergency access to other people.

backed up to 3 separate cloud services

Why so many?

My wife’s parents recently passed. It took months to slog through their stuff and my wife was over it only weeks in. She dumped so much but constantly fights with herself for both taking more than she wanted/needed to and yet less that what she feels she should have. We’ve told our daughter multiple times “our stuff May mean a lot to us, it doesn’t have to mean anything at all to you. If you don’t want it, never feel bad dumping/selling/letting it go.” Out of all the stuff we all collect in life just by living, barely anything has any sentimental value.

On one hand I’ve got a huge collection of photos and albums I’ve taken and collected. I’m trying to clear some out as I go… but I’m not looking forward to that process when my parents go. My dad’s an avid photographer and I know he has a few hundred thousand photos, most of which are near duplicates and he rarely cleans them up.

I already pay for storage for one of them for other reasons (M365), but I much prefer Google Photos as a service, especially for sharing photos and albums, so I pay for that too. The third is just some crazy good deal with "iDrive Photos" where it's $5/year for unlimited upload from my phone haha.

Also in case anything happens to one of my accounts, especially since one is tied to my Xbox account and I have kids.

Does Bitwarden have emergency delegation now? I’d been waiting for it

In 2017, I helped develop key recommendations for planning your digital legacy. These include:

• creating an inventory of accounts and assets, recording usernames and login information, and if possible, downloading personal content for local storage
• specifying preferences in writing, noting wishes about what content should be preserved, deleted, or shared – and with whom
• using password managers to securely store and share access to information and legacy preferences
• designating a digital executor who has legal authority to carry out your digital legacy wishes and preferences, ideally with legal advice
• using legacy features on available platforms, such as Facebook’s Legacy Contact, Google’s Inactive Account Manager, or Apple’s Digital Legacy.

It sounds like something similar to RAG (retrieval augmented generation) or a database lookup. Are you storing the transcripts in a SQL like database or noSQL db or doing semantic similarity on any of it?

I was thinking of a similar project and building a knowledge graph for each person.

Nicceeeee! Thank you!

Thanks to capitalism, you don’t own most of that “digital legacy” and do not have the right to bequeath or transfer ownership for the vast majority of it.

This is stuff that should be around for many decades

Should it? 99.99% of my email doesn't need to be around for more than a few days, let alone decades. And that number will only go up when I'm dead. Really important stuff, like ownership titles, is on file in paper here in my house and with the relevant title agency.

No, we do have that. Social media is a gold mine for analysis, both for modern sociology and for future archaeology.

At least for the 2 years I'm using it

A couple years ago, I would have agreed. Most of our email is junk. But nowadays, you can have an LLM digest and summarize it for you. That could also be a service the legacy system offers.
Grandkids can just ask for a free-form search term without having to wade through everything.

You can take ownership of a lot of it. Thanks to GDPR, major platforms offer ways to export data like photos, videos, activity on their platforms, messages etc. Store locally first, avoid over reliance on online platforms for safekeeping your data.

Also, we need to fight to keep ownership of digital media while we still can. Buy movies and music on physical media so they keep making them. Buy physical books. Buy from DRM free platforms like GoG. As convenient as it may be, avoid over reliance on streaming services.

And of course, make backups of anything you care about. Only you can keep your data safe. Online services will only keep your data as long as they can exploit it to make money.

If you’re interested in “chatting” with your writing there’s a couple of out of the box solutions right now, like Kortex or Reflect Notes. They are AI first note taking apps. I don’t use them out of privacy concerns but if you don’t care that much they might allow you to do what you want. They claim to be E2E encrypted and the AI unable to phone home but these are companies that sprung out of nowhere so I don’t trust they necessarily have done all their homework to actually provide full privacy.

Alternatively there’s an Obsidian plugin that I believe allows you to do such a thing as well with local LLMs if you wanted to which is the privacy first way to this. I’ve just moved to Obsidian from Capacities so I have yet to try it out as I’m still setting up my vault.