API: How can I draw an image over the video in a filter plugin without effect?

[qis]

Hello!

Currently, I have a working filter plugin that renders the frames into a gs_stagesurf_t, modifies the image in RAM, and draws the original video with obs_source_video_render.

How can I copy the modified frame from RAM to VRAM and draw it over the original video?

So far, I've tried creating a texture with gs_texture_create and drawing it with obs_source_draw before or after the obs_source_video_render call, but that does nothing.

I'm aware that the image mask plugin does something similar, but don't understand why it uses an effect shader. Is it possible to just draw a texture over the video?

Here is a simplified version of the plugin:

namespace scan {

constexpr uint32_t sx = 768;   // X offset for the 1024x1024 RGBA image
constexpr uint32_t sy = 28;    // Y offset for the 1024x1024 RGBA image
constexpr uint32_t sw = 1024;  // 1024x1024 RGBA image width
constexpr uint32_t sh = 1024;  // 1024x1024 RGBA image height
constexpr uint32_t dw = 512;   // 512x512 GRAY image width
constexpr uint32_t dh = 512;   // 512x512 GRAY image height

}  // namespace scan

class plugin_impl {
public:
  plugin_impl(obs_source_t* context) noexcept : source_(context)
  {
    scan_.resize(scan::dw * scan::dh);
    overlay_.resize(scan::sw * scan::sh * 4);
  }

  ~plugin_impl()
  {
    if (stagesurf_) {
      gs_stagesurface_destroy(stagesurf_);
    }
    if (texrender_) {
      gs_texrender_destroy(texrender_);
    }
  }

  void render() noexcept
  {
    const auto target = obs_filter_get_target(source_);
    if (!target) {
      obs_source_skip_video_filter(source_);
      return;
    }

    const auto cx = obs_source_get_width(target);
    const auto cy = obs_source_get_height(target);
    if (!cx || !cy) {
      obs_source_skip_video_filter(source_);
      return;
    }

    if (!stagesurf_) {
      stagesurf_ = gs_stagesurface_create(scan::sw, scan::sh, GS_RGBA);
      if (!stagesurf_) {
        return;
      }
    }

    if (!texrender_) {
      texrender_ = gs_texrender_create(GS_RGBA, GS_ZS_NONE);
      if (!texrender_) {
        return;
      }
    }

    bool overlay = false;
    gs_blend_state_push();
    gs_blend_function(GS_BLEND_ONE, GS_BLEND_ZERO);
    gs_texrender_reset(texrender_);
    if (gs_texrender_begin(texrender_, scan::sw, scan::sh)) {
      gs_projection_push();
      gs_ortho(
        float(scan::sx),
        float(scan::sx + scan::sw),
        float(scan::sy),
        float(scan::sy + scan::sh),
        -100.0f,
        100.0f);
      obs_source_video_render(target);
      gs_projection_pop();
      gs_texrender_end(texrender_);

      uint32_t line = 0;
      uint8_t* data = nullptr;
      gs_stage_texture(stagesurf_, gs_texrender_get_texture(texrender_));
      if (gs_stagesurface_map(stagesurf_, &data, &line)) {
        // Apply filter to the 1024x1024 RGBA image and store it in a 512x512 GRAY image.
        scan::filter(data, scan_.data());

        // Create a copy of the original data.
        std::memcpy(overlay_.data(), data, scan::sw * scan::sh * 4);

        // Overlay the 512x512 GRAY image as RED over the copy of the original data.
        scan::overlay(scan_.data(), overlay_.data(), 1.0f, 0.0f, 1.0f, 0.5f);
        overlay = true;

        gs_stagesurface_unmap(stagesurf_);
      }
    }
    gs_blend_state_pop();

    obs_source_video_render(target);

    if (overlay) {
      const uint8_t* data[] = { overlay_.data() };
      if (auto texture = gs_texture_create(scan::sw, scan::sh, GS_BGRA, 1, data, 0)) {
        //gs_matrix_push();
        //gs_blend_state_push();
        //gs_blend_function(GS_BLEND_ONE, GS_BLEND_INVSRCALPHA);
        obs_source_draw(texture, 0, 0, scan::sw, scan::sh, false);
        //gs_blend_state_pop();
        //gs_matrix_pop();
        gs_texture_destroy(texture);
      } else {
        log("could not create overlay texture");
      }
    }
  }

private:
  obs_source_t* source_;
  gs_texrender_t* texrender_{ nullptr };
  gs_stagesurf_t* stagesurf_{ nullptr };

  std::vector<uint8_t> scan_;
  std::vector<uint8_t> overlay_;
};

 

[qis]

I solved my problem by using an .effect file. It would still be interesting to know if it's possible to do this without one.

 

[qis]

Hi this is Gulshan Negi
Well, using Overlay the image using a compositing operation you can draw an image over the video in a filter plugin without effect.
Thanks

Hi Gulshan! Thanks for the suggestion. Are you aware of any plugin that does that? I can't find one that sets the OBS_SOURCE_COMPOSITE flag and does something similar to what I want without using an effect. Or did you mean something different entirely?