Offline Meeting Transcription and Summarization

TL;DR: we are playing with MacWhisper to record, transcribe, and summarize our meetings.

You might be like me with a below-average attention span. Or you might be struggling with keeping up with taking notes and being fully involved in the conversation at hand. I value having notes after a meeting and a way to keep track of different key points and to-do items.

Since most of our meetings are on Zoom, I've looked at a couple services like Fathom AI or Otter AI. Ultimately I'm not sure about trusting a third party with a lot of company and personal data.

Ideally, I want something that is 100% offline, which guarantees privacy and security.

That's when I found MacWhisper. It is great for recording offline meetings with the microphone, and it will prompt you to record when it detects a Zoom meeting starting. Just be sure to get permission from all participants first.

When the meeting is done, it uses a Whisper model to transcribe the meeting with differentiating speakers (diarization). This gives us something that looks like:

Speaker 1
Hello, how is everyone doing?

Speaker 2
Thanks Bob. I'm well.

It doesn't always get everything perfectly in the transcription or identifying which speaker said what. I find it good enough, and it will only get better.

Summarization

MacWhisper has a section for using AI to summarize the meeting. To do it you can hook it up to ChatGPT or Gemini, etc. We've decided we don't want to be uploading our company thoughts and meetings to AI at this point. So, this needs to happen securely.

MacWhisper lets you use an AI hosted locally on the computer. This works perfectly when set up with LM Studio. After the meeting is transcribed, I hit the "Summarize" button and have the key points and takeaways for quick review.

Then I share the summary and transcript with my team.

Next is to figure out how to use AI to ask questions across all the meetings that have happened.