LocalVocal: Local Live Captions & Translation On-the-Go

LocalVocal: Local Live Captions & Translation On-the-Go v0.3.6

royshilkrot

Member
Is it possible to get a Whisper Model for German speech and translate it to show English subtitles?
German should work pretty good with the standard "Tiny" or "Small" models, since it's a well used language with a lot of data

For realtime translation I can suggest
- https://obsproject.com/forum/resources/text-source-translation.1496/ - cloud translation
- or my new Polyglot https://obsproject.com/forum/resources/polyglot-real-time-local-translation-service-for-obs.1818/ - but this will add CPU usage which may not work well if you're already running Whisper on your CPU
 

OtterNas3n

New Member
German should work pretty good with the standard "Tiny" or "Small" models, since it's a well used language with a lot of data

For realtime translation I can suggest
- https://obsproject.com/forum/resources/text-source-translation.1496/ - cloud translation
- or my new Polyglot https://obsproject.com/forum/resources/polyglot-real-time-local-translation-service-for-obs.1818/ - but this will add CPU usage which may not work well if you're already running Whisper on your CPU
Will definitely have a look on the realtime translation stuff you mentioned
Is there a resouce for the Tiny or Small models for German? Having a hard time finding them
 

royshilkrot

Member
Will definitely have a look on the realtime translation stuff you mentioned
Is there a resouce for the Tiny or Small models for German? Having a hard time finding them
What i meant was just the "built in" multilingual Tiny and Small should be fine. Just not the ".en" models, those are just for English.
There's this for example https://huggingface.co/daveni/whisper-tiny-commonvoice_v11-de/tree/main which you'd need to convert to GGML with the tools provided in https://github.com/ggerganov/whisper.cpp
 

andersonid

New Member
[ Feature Request ] - Please, mate! Your work is awesome! I can buy you a beer or a juice and help with some donation... but, please take some time to add a RPM version for linux, or some instructions on how to build from source code for others distros. I'm not a developer and the CI script provided doesnt work on Fedora/RHEL. I cant figure out how to fix each error message presented.
 

royshilkrot

Member
[ Feature Request ] - Please, mate! Your work is awesome! I can buy you a beer or a juice and help with some donation... but, please take some time to add a RPM version for linux, or some instructions on how to build from source code for others distros. I'm not a developer and the CI script provided doesnt work on Fedora/RHEL. I cant figure out how to fix each error message presented.
Thanks for the kind words! Your support keeps me going

I'll happily do it, in fact in my other plugin I already have some scripts so I can reuse them.
If you have GitHub account can you open an issue for it? Just so I have it on my to-do list https://github.com/occ-ai/obs-localvocal/issues
 

royshilkrot

Member
royshilkrot updated LocalVocal: Seamless Live Transcriptions On-the-Go with a new update entry:

v0.0.7 - ~25% performance increase! and fixing crash bug

This release bumps Whisper.cpp to a version that brings ~25% gains in performance (!), and also fixing the notorious "remove filter crash" bug.
Let me know if you experience any issues!

If you'd like to support my work consider Patreon or GitHub Sponsors, a ⭐ on the repo would be greatly appreciated and a rating on the...

Read the rest of this update entry...
 

jarnach

New Member
Hello! First of all, I would like to thank the creators for their time spent on the project!
I've noticed a problem with translating into Polish. Instead of the Polish "Ż", the Czech "Ž" appears.
Other Polish characters display correctly. Best regards :)
 

adamesek

New Member
Hello! First of all, I would like to thank the creators for their time spent on the project!
I've noticed a problem with translating into Polish. Instead of the Polish "Ż", the Czech "Ž" appears.
Other Polish characters display correctly. Best regards :)
a.. działa Tobie poprawnie z językiem polskim? Pytam o sam LocalVocal. U nas ..idzie.. ale znaków typu: ś, ć itp. nie widać całkowicie /obs na mac/. Pozdrawiam.
 

jarnach

New Member
a.. działa Tobie poprawnie z językiem polskim? Pytam o sam LocalVocal. U nas ..idzie.. ale znaków typu: ś, ć itp. nie widać całkowicie /obs na mac/. Pozdrawiam.
tak jak pisałem oprócz Ż wszystko działa. oczywiście jakość tłumaczenia to inna sprawa :)

v0.0.7 windows 10​

 

1cor1613

New Member
I'm trying to use this to create a transcription in a file of a video. I'd like to see all the transcription output in a file that I can edit or place into a word editor. The Text file seems to overwrite itself, but I need the running dialogue. I also don't want to have to weed out a lot of timestamps throughout. Is there a way to get a text output like I'm looking for?
 

royshilkrot

Member

Handy Andy

New Member
Anyone have luck installing and using LocalVocal OBS plugin for captioning? I'm using OBS 30.02 with LocalVocal 1.0 and it's not working. I can add it to audio sources as a filter, but have no preview and can't find how to enable/disable it. It does not show up in the Tools drop-down? When I used it with OBS 29.0.2 I had it partially working with the preview screen but not live, and the enable/disable option, but it did not show up as an audio filter?
 

royshilkrot

Member
Anyone have luck installing and using LocalVocal OBS plugin for captioning? I'm using OBS 30.02 with LocalVocal 1.0 and it's not working. I can add it to audio sources as a filter, but have no preview and can't find how to enable/disable it. It does not show up in the Tools drop-down? When I used it with OBS 29.0.2 I had it partially working with the preview screen but not live, and the enable/disable option, but it did not show up as an audio filter?
@Handy Andy we can support at https://discord.gg/CZBHWUTwts
maybe LV is working just not sending output to any text source?
let us see some logs etc. and we can try to find a solution
 

wombat94

New Member
This plugin is awesome. It is so great to have a CC plugin with local recognition and stream output.

But, it suffers from the same challenges as the other caption streaming implementations when it comes to YouTube. YT doesn't like the way in which obs_output_output_caption_text1 puts the 608 captions in the stream. Consistency of captions actually appearing is nonexistent and for the ones that do appear they can easily become minutes behind for a stream that is more than just a few minutes long itself.

After researching and seeing that WebCaptioner had experimental support for sending captions via HTTP post to YT for live captioning (which the few references I could find to it from actual users was working very well). I whipped up a VERY crude/rudimentary POC this morning by creating a python script that ran a loop tailing the local log file from LocalVocal's transcript and sends the captions to YT along side the stream that OBS is sending.

It works, but obviously it is not something that I'd want to use even or our own church services.

Would you be interested in incorporating an option to use the YT http POST method for closed captioning alongside the OBS WebSocket option you have now?

If not, I'll probably pursue some other method, but I KNOW it would be huge for many people to have it integrated and it is NOT a huge amount of effort.

My basic method was about 20 lines of python code (and that includes the code to tail the log file you are writing out).

I'm going to post an issue to your GitHub for the project. (And though I am not a C++ person by trade, I may take a show at a contribution to enable this).
 

royshilkrot

Member
This plugin is awesome. It is so great to have a CC plugin with local recognition and stream output.

But, it suffers from the same challenges as the other caption streaming implementations when it comes to YouTube. YT doesn't like the way in which obs_output_output_caption_text1 puts the 608 captions in the stream. Consistency of captions actually appearing is nonexistent and for the ones that do appear they can easily become minutes behind for a stream that is more than just a few minutes long itself.

After researching and seeing that WebCaptioner had experimental support for sending captions via HTTP post to YT for live captioning (which the few references I could find to it from actual users was working very well). I whipped up a VERY crude/rudimentary POC this morning by creating a python script that ran a loop tailing the local log file from LocalVocal's transcript and sends the captions to YT along side the stream that OBS is sending.

It works, but obviously it is not something that I'd want to use even or our own church services.

Would you be interested in incorporating an option to use the YT http POST method for closed captioning alongside the OBS WebSocket option you have now?

If not, I'll probably pursue some other method, but I KNOW it would be huge for many people to have it integrated and it is NOT a huge amount of effort.

My basic method was about 20 lines of python code (and that includes the code to tail the log file you are writing out).

I'm going to post an issue to your GitHub for the project. (And though I am not a C++ person by trade, I may take a show at a contribution to enable this).
Yes I can add that to the plugin. Thank you very much for the snippet, it will help in making the C++ integration. And thanks as well for the suggestion on how to make it better!
 
Top