[BlindMath] Semantic Digital Audio Memory: A cognitive aid to boost the capabilities of your memory

[Rastislav Kish]

Hello list,
I would like to share with you a project I had in mind for longer time during my university study, and which I finally got to work on in the recent months.
While attending classes of theoretical mathematics, I’m usually facing 3 problems:

- I can’t write down notes and pay attention at the same time
- Sometimes, I don’t get the context of the explained concept right away, I need few moments to think it through or even lookup additional details in my notes or on the Internet. So, I either don’t do so and end up just sitting in the class being unable to understand anything, because that concept was important for later topics, or, I do the lookup asynchronously, what however means I get out of sync with the explanation and find myself in the same situation, except now I can’t do much with it.
- If the class requires active work, my mind gets submerged in the problem and can’t track anything in the physical world, resulting in shattered context and missed information.

Recording classes can fix all of these issues, however for the cost of doubling the processing time for each class, since raw recordings don’t hold any information about their content and need to be listened through in full to get a good-quality notes.

Semantic audio

SDAM lets you capture recordings with assigned meaning. In the simplest usage, you can just start the recording and add a mark whenever something you will want to write down later is said, when the class is over, you can just return to those labels and quickly create the notes, you can be sure you have covered everything important without the need to go through the whole thing again. At the same time, those marks can serve as reference points, if you need to return in your memory to the part of your class dealing with a particular topic, because you feel you may have missed something or just want to hear it again, you can get to the relevant part in few clicks.

Time travel

However, SDAM also offers a different operation mode. If you have headphones with active noise cancellation technology, you can use it to travel in time during the class. After activating this function, the program will work in augmented reality mode, where you can hear what’s happening around you. And if you don’t get something, need to research or simply mishear, there’s nothing simpler than pausing the time or rewinding it back, you will get to repeat the past events without missing on anything that’s happening in the meantime, because everything is being recorded for you in the background. So when you’re done, you can simply continue listening to the class as it was happening while you were dealing with other things, or, even increase the speed twice or triple to get in sync again.

The program is also equipped with a built-in notepad, so you can make use of it to do your note-taking stuff, calculations and other textual operations.

Saving your memory to a file

When the class is over and you save everything, all the recorded audio, taken marks and written notes is put into a single file, which can be afterwards opened again in SDAM and act as a effective capture of your memory back from the class.

This project is highly experimental, I’ve got all of the above mentioned implemented, and I’m curious to see how are my ideas going to work in practice. Over the time, I would also like to add more functionality related to audio processing, like automatic transcription using Whisper (that of course won’t work for math, but could give a decent enough starting point for more narrated topics), automatic silence detection and removal (combined with timetravel, that could be a really interesting function), and I have more cool stuff in mind. The idea is basically that SDAM could become my all-in-one solution for working with audio classes, increasing effectivity and saving time for more of the fascinating topics.

If you find the idea interesting, you can learn more about the project in it’s [GitHub repository](https://github.com/RastislavKish/sdam). It’s free and open-source, as usual with my projects.

Happy memory-hacking!

Best regards

Rastislav

​