LocalVocal: Local Live Captions & Translation On-the-Go

LocalVocal: Local Live Captions & Translation On-the-Go v0.3.9

Go to download
Overview Updates (22) Reviews (9) History Discussion
royshilkrot

royshilkrot

Member
royshilkrot updated LocalVocal: Local Live Captions & Translation On-the-Go with a new update entry:

v0.3.4 AMD GPU! CoreML and Metal! Accelerate!

Much rejoicing! Introducing AMD ROCm / hipBLAS acceleration, as well as CoreML and Metal on Apple!
This was tested to give tremendous performance boosts on Apple and AMD GPUs.

Also in this release:
  • bumping to the latest whisper.cpp
  • fixing the captions stream issue on Twitch and YouTube
  • fixing the translation captions disappearance.
Enjoy and let me know of any problems!

Download:
  • ...
Click to expand...

Read the rest of this update entry...
 
Q

quyhp

New Member
Hi! I've installed obs-localvocal for windows (Windows 11 - with CUDA) to C:\obs-studio but I can't find any executable file in there or in any subfolder, so how can I run the app?
 
royshilkrot

royshilkrot

Member
quyhp said:
Hi! I've installed obs-localvocal for windows (Windows 11 - with CUDA) to C:\obs-studio but I can't find any executable file in there or in any subfolder, so how can I run the app?
Click to expand...
This is an OBS plugin and it will appear on OBS as an audio filter on any audio source
 
R

ryancy

New Member
thank you for your gerat work, roy
I have a problem that {{input}} doesn't capture Dynamic input from Localvocal subtitles.
Could you solve this?
スクリーンショット 2024-08-05 191221.png
 

Attachments

  • スクリーンショット 2024-08-05 191247.png
    スクリーンショット 2024-08-05 191247.png
    55.7 KB · Views: 22
  • スクリーンショット 2024-08-05 191333.png
    スクリーンショット 2024-08-05 191333.png
    44 KB · Views: 18
Brantyz

Brantyz

New Member
Плагин отличный, но есть минус в том что я настроил обс по кнопке говорить, а этот плагин захватывает звук даже когда я не нажимаю на кнопку если вы добавите функцию, исправляющую эту проблему было бы круто и я бы смог использовать ваш плагин!
The plugin is great, but the downside is that I have customized the obs by talk button and this plugin captures the sound even when I don't click the button if you add a feature fixing this problem it would be cool and I would be able to use your plugin!
 
royshilkrot

royshilkrot

Member
T

Tallicia

Member
What is required to run with the hipBLAS 0.3.5 installation? When I try to use it, OBS won't start. The tail of the log doesn't offer much. It appears it causes OBS to crash. My only solution is to install the CPU version so it stops crashing and will start up.

19:56:23.674: [obs-localvocal] Checking if model 'Whisper Medium En q5' exists in data...
19:56:23.674: [obs-localvocal] Model not found in data: C:\Program Files\obs-studio\data\obs-plugins\obs-localvocal\models\ggml-model-whisper-medium-en-q5_0
19:56:23.674: [obs-localvocal] Config models folder: \AppData\Roaming\obs-studio/plugin_config/obs-localvocal/models
19:56:23.674: [obs-localvocal] Checking if model 'Whisper Medium En q5' exists in config...
19:56:23.674: [obs-localvocal] Lookig for model in config: \AppData\Roaming\obs-studio\plugin_config\obs-localvocal\models\ggml-model-whisper-medium-en-q5_0
19:56:23.674: [obs-localvocal] Model folder exists in config folder: \AppData\Roaming\obs-studio\plugin_config\obs-localvocal\models\ggml-model-whisper-medium-en-q5_0
19:56:23.674: [obs-localvocal] Model bin file found in folder: \AppData\Roaming\obs-studio\plugin_config\obs-localvocal\models\ggml-model-whisper-medium-en-q5_0\ggml-model-whisper-medium.en-q5_0.bin
19:56:23.870: [obs-localvocal] Loading whisper model from \AppData\Roaming\obs-studio\plugin_config\obs-localvocal\models\ggml-model-whisper-medium-en-q5_0\ggml-model-whisper-medium.en-q5_0.bin
19:56:23.871: [obs-localvocal] Using hipBLAS for inference
19:56:23.871: [obs-localvocal] DTW token timestamps disabled
Click to expand...

I have and AMD cpu and AMD GPU and am hoping to improve efficiency and get closer to real time Captions.

CPU version log shoes it's much happier
20:02:50.642: [obs-localvocal] Using CPU for inference
20:02:50.642: [obs-localvocal] DTW token timestamps disabled
20:02:51.138: [obs-localvocal] Whisper model loaded: AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0 | CANN = 0
20:02:51.171: [obs-localvocal] Whisper context is null, exiting thread
20:02:51.324: [obs-localvocal] Loading whisper model from
20:02:51.324: [obs-localvocal] Using CPU for inference
20:02:51.324: [obs-localvocal] DTW token timestamps enabled
20:02:51.324: [obs-localvocal] Failed to open whisper model file
20:02:51.324: [obs-localvocal] Failed to initialize whisper context
Click to expand...
 
T

Tallicia

Member
Great update It seem to be getting more and more responsive and a slightly better accuracy.

I still have one constantly reoccurring issue.

I am using the Whisper Tiny English q8 ( 42mb) and the update from today v0.3.8- It keeps hallucination the word "you" It does it right away it seem like at initialization and after I stop talking at least 90% of the time.

Is there anything I can do to make this stop or any idea what might cause this?!
 
T

Tallicia

Member
@royshilkrot Are there instructions for using the vulkan or hiblas versions to have GPU inference versus CPU?

Everything I try results in CPU inference use.

Also many of the models in the list do not download and break the local vocal from working.

AppData\Roaming\obs-studio\plugin_config\obs-localvocal\models\ggml-model-whisper-base-en-q5_1\ggml-model-whisper-base.en-q5_1.bin
12:23:51.458: [obs-localvocal] Using CPU for inference
12:23:51.458: [obs-localvocal] DTW token timestamps enabled
12:23:51.535: [obs-localvocal] Whisper model loaded: AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0 | CANN = 0
Click to expand...
 
LordRouter

LordRouter

New Member
Hi,
downloaded and installed hipblas and cpu only versions:

The 1st one crash after selecting audio source-->Filters-->LocalVocal, no error messages at all.
The 2nd one work, but not as espected because the translation is not a correct translation, every word translated is a non-sense word (or sentence).
Tiny English 74Mb and Small English 495Mb tried.
Model M2M-100 418mb.

Input language italian, output english.

Speaking in english also works but no translation were made.

OBS 30.2.3

Win 11, Asus Prime z390-A, i5 9600k, GPU rx6750xt, 16gb ram, nvme 1tb.
 
You must log in or register to reply here.
Top