Specialized Encoder for 2D Pixel-based/Retro games

[mmassi]

My suggestion (sorry for limited knowledge of the matter)
Is it possible to have specialized encoding for Graphics thats based on pixel art?

What i mean is the video equivalent of jpeg compression vs PNG compression. Alot of old games and retro games that are streamed (esp. in the Speedrunning and lets-play communities) look alot better when no unintentional anti-aliasing is introduced by the compression. Often times old games are also scaled using SAI filters etc, which only look as intended if no additional blending of neighboring pixels occures.

Do others see the need for such a thing? Does something like this exist allready? Is compression like this possible for video at all?

Thanks for any thoughts =D

 

[paibox]

The Flash stream player you see on Twitch and other streaming services does not support any codec that is more useful than the one currently being used, which is h.264. There are ways to make low resolution console games (especially emulated) look better on stream, typically by tweaking the deblock settings for the encoder.

As for 2xSAI filters and the alike, you can't really do anything about that... well, you could turn it off, since it tends to look absolutely horrible. :P Anyway, while compression formats like these may "exist" (such as the thing DOSbox uses, I guess), they can't be used for streaming, it's that simple.

 

[mmassi]

so youre saying unless Adobe or Twitch realize the relevance of retro and indy gaming a full solution wont be possible? =D
I mean for twitch its not even unrealistic even to think about it, if it would allow for high quality 2D game stream with low cpu and bitrate needs.

As an immediate solution, what setup/settings would you recommend for best results? I tried blowing up my emulator to 4x (which is lossless obviously), resulting in a say 1024x768pixels then streaming that in half that resolution, which is fine for the most part but pixel-perfect obviously looks better than something resampled.
The worst alternative bit-rate wise is obviously to stream in some rediculously high resolution to oversample the low-res console graphics.

Do you know if a pixel-art specific codec would actually be anything efficient? Say if a speedrunning site where to implement their own player.

Edit: Revelant for people interested in the topic: http://ingomar.wesp.name/2011/04/dosbox ... pture.html

Edit: Ive read that "x264 lossless" is part of the H264 standard? so it should be actually not a problem at all, if the streaming software would encode accordingly?

 

[paibox]

The codec DOSbox uses is the only thing I know of that performs decently for stuff like low resolution 2D pixel art graphics, but yes, either Adobe or Twitch would have to support such a codec, and there is currently no efficient way to implement additional encoders on the application side (which would be required, unless someone made a plugin for it, which would also be hard right now, but will probably be a lot easier in the future).

The problem with scaling is that you should ideally stream at the resolution you're playing the game.
This video here is done using a capture card with RGB input, but the same thing applies to emulators, really:
http://www.twitch.tv/paizuribox/b/377730269
(Emulators typically look slightly better, since it's always a clean signal.)

For something similar to that, you can add "deblock=1:1" (without the qoutes) to your Custom x264 Encoder Settings under the "Advanced" section of the OBS settings. This fights the "pixel puke" that often occurs when 2D games like these scroll around a lot, though it may cause large color blocks if you don't have the bit rate set high enough.

 

[mmassi]

Ninja Edit during your reply:

Ive read that "x264 lossless" is part of the H264 standard? so it should be actually not a problem at all, if the streaming software would encode accordingly?

x264 is nativly supported by ZSNES for instance. So there is a chance or is there?

Edit: Watched your video and its supersmooth, not perfect perfect pixels but pretty damn close =D What makes it so beatiful to watch is the high FPS though <3 thanks for help =D

 

[Muf]

The easiest way is to add "no-deblock=1" to your custom x264 settings.

 

[paibox]

While x264 supports lossless just fine, and OBS does support using it, you would never be able to stream using anything close to lossless, it would simply use too much bandwidth. For now, I would say just go with a realistic solution. :)

 

[mmassi]

ive done some quick math: EDIT SPOILER BECAUSE MATH ALL WRONG =D

Spoiler

take a 30FPS NES game (30x256x224x8bit) => 1,680kpbs raw. Even with naiv png compression would result in "only" 800kps, with actually video compression even lower so quite feasible or am i missing something?

EDIT: Math correction: its actually 13440 kbit per second raw but png compression is 1:5 on average, so 2688kbps naiv compressed, so would be under 1,5kpbs if the video-effect in the compression is more than 1:2 which seems reasonable too assume. /EDIT

The easiest way is to add "no-deblock=1" to your custom x264 settings.

thanks will try that =D

I see you follow feasel stream, imagine the encoder would analyse his captured video to match actual pixels and you could get godlike quality out of a captured signal no? =D

 

[paibox]

Edit: Alright, so you realized your math was off already, but you still need to stop using "b" (bit) as a substitute for "B" (byte), because it makes even less sense now. :)

Either way, that video I linked you earlier, that was 1600Kbps (bit, not byte) at 60 frames per second. You really don't need this lossless business for streaming, and it's very impractical to begin with, I would suggest using it for recording instead.

Also, I do believe that parsing pixel data out of composite signals is just asking for trouble, sorry. :D

 

[mmassi]

i think you are the one confusing letters now? k or K is no difference is it? i use small b for bit and B for Byte.
so there should be kBps and kbps or not? you say you mean bit but use a capital K? not sure what the conventions are there.
Checked what i wrote and think its fine but its not the point anyway =D you have to test it anyway to see if its practical.
xplit gives bitrates in kbps as far as im aware (1024 bit per second) and my ISP speedtest too so those are the figures im referring too =D

i think upping video quality for composite signals that are allready digitialized anyway will come anyway sooner or later because retro gaming is here to stay and viewer numbers are growing

you say its not really needed but upping video quality just by using appropriate methods is a valid concern imo. Not everybody obsesses about pixels being real pixels but many do =D

if theres any way OBS could support those type of graphics it would give them an edge, as indie games are usually developed with pixelart and retrogaming is ever present

 

[Muf]

If you want higher quality retro gaming capture, use RGB mods on your consoles and get a proper capture card like the StarTech PEXHDCAP. Even the NES can be RGB modded. There is no reason to stick with composite.

 

[Bensam123]

While x264 supports lossless just fine, and OBS does support using it [...]

Are you talking about using crf=0 or is there something else?