video-v1/vav2/docs/working/mediacodec-recreation.md

MediaCodec Recreation Issue Analysis

Why MediaCodec Recreation is Needed

Problem

MediaCodec is initially configured with surface=nullptr (CPU mode), but later we need it to output to ImageReader surface (GPU mode) when Vulkan device is registered.

Android MediaCodec Limitation

AMediaCodec_setOutputSurface() API has limitations:

• Only works for video tunneling mode
• Does NOT work reliably with async mode callbacks
• Cannot change surface after codec is started in async mode

Current Flow

1. Initialize() creates MediaCodec with surface=nullptr
2. MediaCodec starts with async callbacks
3. SetVulkanDevice() is called later
4. ImageReader surface is created
5. Need to switch MediaCodec output to ImageReader surface
6. setOutputSurface() doesn't work in async mode → Must recreate codec

Current Issue After Recreation

Even after recreation, getInputBuffer() returns null for buffer indices from callbacks.

Root Cause

Race condition after AMediaCodec_start():

• Callbacks fire immediately when codec starts
• Buffer indices are enqueued (0-9)
• BUT codec internal buffer allocation isn't complete yet
• When getInputBuffer() is called with these indices → returns null

Evidence from Logs

23:32:11.938 - Codec started, callbacks fire (indices 0-9 queued)
23:32:12.142 - DecodeFrameAsync gets index 0
23:32:12.142 - getInputBuffer(index=0) → returns null!

Possible Solutions

Option 1: Wait after start() then clear queue

After AMediaCodec_start(), wait 10-50ms for codec to stabilize, then clear old indices and wait for new callbacks.

Option 2: Retry logic

If getInputBuffer() returns null, put index back in queue and retry with next index.

Option 3: Don't recreate - use software decode path

Accept that async mode with surface switching doesn't work well, fall back to CPU decoding.

Option 4: Change initialization order (BEST)

• Call SetVulkanDevice() BEFORE Initialize()
• But this requires changing application code

Recommendation

Try Option 1 first - it's the simplest fix:

// After AMediaCodec_start()
std::this_thread::sleep_for(std::chrono::milliseconds(50));
// Clear old buffer indices
{
    std::lock_guard<std::mutex> lock(m_async_handler->m_async_mutex);
    while (!m_async_handler->m_async_input_buffer_queue.empty()) {
        m_async_handler->m_async_input_buffer_queue.pop();
    }
}
// Wait for new callbacks with valid buffers