LocalVocal: Seamless Live Transcriptions On-the-Go

LocalVocal: Seamless Live Transcriptions On-the-Go v0.1.0

appa561

New Member
@corint1 i'm sorry this is the experience you got. do you want to help debug by providing logs and such?
you can open an issue https://github.com/royshil/obs-localvocal/issues and send the log files there
to find log files is simple e.g. https://help.uscreen.tv/en/articles/4747667-live-stream-troubleshooting-how-to-download-obs-logs
I think I may have had a similar issue to corint1. When I initially installed the localvocal, it worked, as I indicated early. I have no idea what triggered it to stop working. When I attempted to restart OBS, nothing. OBS wouldn't load. Uninstalling LocalVocal via Windows allowed OBS to start up again.

Today, I have other tasks to attempt to accomplish before I stream tonight, so no time available to do testing and log grabbing... Maybe Thursday but more realistically next week's days off from the paying job will give some time to pursue the issues via logs.
 

royshilkrot

Member
I think I may have had a similar issue to corint1. When I initially installed the localvocal, it worked, as I indicated early. I have no idea what triggered it to stop working. When I attempted to restart OBS, nothing. OBS wouldn't load. Uninstalling LocalVocal via Windows allowed OBS to start up again.

Today, I have other tasks to attempt to accomplish before I stream tonight, so no time available to do testing and log grabbing... Maybe Thursday but more realistically next week's days off from the paying job will give some time to pursue the issues via logs.
@appa561 I'm sorry LocalVocal caused a crash. I'm interested in the logs and crash logs/report to see where the problem lies
I see you already opened an issue with logs and I'm taking a look. https://github.com/royshil/obs-localvocal/issues/20
 

Krakhen

New Member
I really like where this plugin is going, I tried it out today during the first minutes of a BF2042 stream and even though I believe "Translation" is still marked as a feature coming in the future it's the one I personally find to have the most potential if my PC is capable of running the models.
In the end I decided to disable it because of the in-game performance since my setup doesn't seem capable of handling the load.
I find "Tiny" to be pretty inaccurate, "Small" does a much better job.
About the first 20 minutes of this stream have the plugin enabled: https://www.twitch.tv/videos/1929612653
 

royshilkrot

Member
I really like where this plugin is going, I tried it out today during the first minutes of a BF2042 stream and even though I believe "Translation" is still marked as a feature coming in the future it's the one I personally find to have the most potential if my PC is capable of running the models.
In the end I decided to disable it because of the in-game performance since my setup doesn't seem capable of handling the load.
I find "Tiny" to be pretty inaccurate, "Small" does a much better job.
About the first 20 minutes of this stream have the plugin enabled: https://www.twitch.tv/videos/1929612653
duuude! this is awesome!! looks really great, and the translation.. sometimes.. it works OK. but i'm glad to see it's working well enough for a stream.
 

royshilkrot

Member
royshilkrot updated LocalVocal - Live stream AI assistant with a new update entry:

v0.0.3 - real-time transcription, stream captions, fix model download

In this release we have several important updates: send captions to an RTMP stream, fixing the model download issues, step-by-step transcription (realtime transcription effect). New version of Whisper.cpp has major speedup on Mac CPUs!

If this plugin has provided value to you consider adding a ⭐ on the GH repo, subscribing to my YouTube channel, and supporting my work...

Read the rest of this update entry...
 

appa561

New Member
Installed the new 0.0.3 version last night using base, while a bit slow, still useable. Opened and closed OBS a few times without issue. Hoping to test on stream tonight to see how it flows. If that works, the only thing left for my use case would be translating to something other than English. I speak English on stream and would like the CC translated to Spanish, for example.
 

royshilkrot

Member
Installed the new 0.0.3 version last night using base, while a bit slow, still useable. Opened and closed OBS a few times without issue. Hoping to test on stream tonight to see how it flows. If that works, the only thing left for my use case would be translating to something other than English. I speak English on stream and would like the CC translated to Spanish, for example.
thanks for testing! let me know how it goes please

i'm working on adding the LLM so you can do translation and other crazy stuff.
it's ... going.
 

AptiX1

New Member
Good day! Thanks for the plugin, but it doesn't work in Russian. Apparently there is a problem with Cyrillic. When the Russian language is exposed, she begins to write texts incorrectly. In OBS, it displays in the form of "???" question marks, and if you try in the output file, then... well, you can see for yourself :D
 

Attachments

  • image_677.png
    image_677.png
    13 KB · Views: 12
  • image_678.png
    image_678.png
    16.3 KB · Views: 13
  • image_679.png
    image_679.png
    9.1 KB · Views: 13

royshilkrot

Member
Good day! Thanks for the plugin, but it doesn't work in Russian. Apparently there is a problem with Cyrillic. When the Russian language is exposed, she begins to write texts incorrectly. In OBS, it displays in the form of "???" question marks, and if you try in the output file, then... well, you can see for yourself :D
@AptiX1 sorry about that. It's fixable. Can you please create an issue so I can track this bug? https://github.com/royshil/obs-localvocal/issues
Add the screenshots as well plz
 

Guuvita

New Member
Ok, 0.0.3 does not crash but all i get are random phrases, like im literally not even talking and im getting random spanish (set to spanish) phrases and well... a lot of � symbols instead of á é í ó ú ñ etc
tried to check the tutorial but the UI is different, no idea what VAD enabled is, and im getting the phrases after disabling that, while enabled there is no text at all
 

royshilkrot

Member
Ok, 0.0.3 does not crash but all i get are random phrases, like im literally not even talking and im getting random spanish (set to spanish) phrases and well... a lot of � symbols instead of á é í ó ú ñ etc
tried to check the tutorial but the UI is different, no idea what VAD enabled is, and im getting the phrases after disabling that, while enabled there is no text at all
yep i'm examining the problem of the unicode characters, i believe this is a windows only problem with string processing (wide-char).

the "leftovers" that you're getting are most likely the result of audio noise
if you can get really crisp audio that would be the best scenario
you can use as a pre-filter the "Noise Suppression" filter built-in OBS. put it above LocalVocal in the filter list so that LV gets a filtered audio only of your speech. that would improve things.
keep the VAD (Voice Activity Detection) on. it will suppress silent parts

i'll be shooting a new tutorial video with extra features soon.
 

appa561

New Member
Have been using it off and on since 0.0.3 came out with the Base model. Accuracy isn't perfect, but not bad at the same time. Viewers are reporting slightly distorted audio (speech) while using the filter. When I turn it off, the problem goes away.

Is there a breakdown of what all the settings are for? I am wondering if I cannot tweak some default setting and get better performance. Admittedly, I am a novice to all the stuff. Is there a way to take advantage of the RTX 3060Ti in this box?
 

royshilkrot

Member
@appa561 i'm interested in knowing what the distortion is, as i couldn't sense it from your Twitch stream
you can submit a GitHub issue for that so I can track the problem

to use your Nvidia card you'd need a CUDA build.
the option is available (https://github.com/royshil/obs-localvocal#building-with-cuda-support-on-windows) but requires a build on your computer since CUDA versions vary greatly from one PC to another. that's why it makes little sense to release it for download
 

appa561

New Member
@appa561 i'm interested in knowing what the distortion is, as i couldn't sense it from your Twitch stream
you can submit a GitHub issue for that so I can track the problem

to use your Nvidia card you'd need a CUDA build.
the option is available (https://github.com/royshil/obs-localvocal#building-with-cuda-support-on-windows) but requires a build on your computer since CUDA versions vary greatly from one PC to another. that's why it makes little sense to release it for download
I went back to the VOD on Twitch and watched a replay from yesterday afternoon. There is definitely some distortion and went away when I turned off LV... I will clip out a representative portion and post it in the issue tracker either tonight or in the morning before I head to work. If neither of those work out, I am off on Friday.
 

Guuvita

New Member
yep i'm examining the problem of the unicode characters, i believe this is a windows only problem with string processing (wide-char).

the "leftovers" that you're getting are most likely the result of audio noise
if you can get really crisp audio that would be the best scenario
you can use as a pre-filter the "Noise Suppression" filter built-in OBS. put it above LocalVocal in the filter list so that LV gets a filtered audio only of your speech. that would improve things.
keep the VAD (Voice Activity Detection) on. it will suppress silent parts

i'll be shooting a new tutorial video with extra features soon.
Nah, 0 noise, i use a dedicated noise supression that is quite good, obs shows no noise at all either, if i remember correctly the text appeared even if i muted the mic (physically)
About the characters, are you using UTF-8? that fixes that issue for websites and for documents on windows
 
Top