H.264 Video Streaming to Twitch via RTMP - Packet Format Issues

Other Questions Twitch
1

I’m building a streaming application that captures screen content and sends it to Twitch using RTMP protocol. The encoded video data gets processed by x264 and then transmitted to Twitch servers. While the servers accept my data stream, the video won’t play properly. It shows a brief loading screen before displaying error 2000 (network error). Here’s my current approach for handling the encoded frames:

int encoded_size = x264_encoder_encode(h264_encoder, &nal_units, &nal_count, &input_pic, &output_pic);

// Process each NAL unit based on its type
int pts_dts_offset = int(output_pic.i_pts - output_pic.i_dts);
pts_dts_offset = htonl(pts_dts_offset);
BYTE *offset_ptr = ((BYTE*)&pts_dts_offset) + 1;

FrameData frame_data;
bool has_video_frame = false;

uint8_t *sequence_params = NULL;
int sps_length = 0;

for (int j = 0; j < nal_count; j++) {
    x264_nal_t &current_nal = nal_units[j];
    
    if (current_nal.i_type == NAL_SPS) {
        sps_length = current_nal.i_payload;
        sequence_params = (uint8_t*)malloc(sps_length);
        memcpy(sequence_params, current_nal.p_payload, sps_length);
    } else if (current_nal.i_type == NAL_PPS) {
        uint8_t *combined_data = (uint8_t*)malloc(current_nal.i_payload + sps_length);
        memcpy(combined_data, sequence_params, sps_length);
        memcpy(combined_data + sps_length, current_nal.p_payload, current_nal.i_payload);
        frame_data = {current_nal.i_payload + sps_length, combined_data, current_nal.i_type};
        video_queue->push(frame_data);
    }
}

The processed data goes into this structure:

struct FrameData {
    int data_size;
    uint8_t* video_data;
    int nal_type;
};

Then my broadcast module sends it via RTMP:

FrameData current_frame = video_queue->front();
video_queue->pop();

RTMPPacket *rtmp_packet = (RTMPPacket*)malloc(sizeof(RTMPPacket) + RTMP_MAX_HEADER_SIZE + current_frame.data_size + 9);

rtmp_packet->m_body = (char*)rtmp_packet + sizeof(RTMPPacket) + RTMP_MAX_HEADER_SIZE;
rtmp_packet->m_nBodySize = current_frame.data_size + 9;

unsigned char *packet_body = (unsigned char*)rtmp_packet->m_body;

packet_body[0] = (current_frame.nal_type == NAL_SLICE_IDR) ? 0x17 : 0x27;
packet_body[1] = 0x01;
packet_body[2] = 0x00;
packet_body[3] = 0x00;
packet_body[4] = 0x00;

memcpy(&packet_body[9], current_frame.video_data, current_frame.data_size);

RTMP_SendPacket(rtmp_connection, rtmp_packet, TRUE);

Twitch receives the stream at about 3kbps but won’t play the video. What could be wrong with my packet formatting?

2

Your RTMP packet’s missing the 4-byte size field that needs to go before your video data. After you set packet_body[4] = 0x00, you’ve got to write the frame size in big-endian format before copying the actual video data.

Add this after packet_body[4] = 0x00:

uint32_t frame_size = htonl(current_frame.data_size);
memcpy(&packet_body[5], &frame_size, 4);

Your timing info’s also messed up. RTMP needs proper timestamps for video sync, but you’re hardcoding zeros in bytes 2-4. That network error 2000 usually means Twitch can’t decode your video packets because they’re malformed. I ran into the exact same problem when I started with RTMP streaming - the missing size prefix was what broke everything.

3

Your code has a fundamental issue with NAL unit handling. You’re only processing frames when you hit a PPS unit, but you need to handle all frame types separately. The real problem is you’re not sending SPS/PPS config data to Twitch correctly. Twitch wants the decoder config record (with SPS/PPS) sent as a separate RTMP packet using packet_body[1] = 0x00, not 0x01. Send this config packet before any video frames. Right now you’re combining SPS/PPS with regular frames, which breaks the decoder. You’re also missing composition time offset calculation - even if you set it to zero, timestamps need proper handling. That 3kbps bitrate is probably too low too. Twitch usually needs higher bitrates for stable streaming.