Jump to content

byte-range requests for podcast mp3


benbyf
 Share

Recommended Posts

related to this topic on wireSendFile(). I've been trying to create a RSS feed for a podcast which i've done previously using PW 2.7.2 with no trouble (e.g. http://machine-ethics.net/), however sitting on the same infustructure a new site (http://www.threepointspodcast.com/) on PW 3.0.42 is having issues validating with itunes with this error from them: Can’t submit your feed. Your episodes are hosted on a server which does not support byte-range requests. Enable byte-range requests and try again. Which is obviously untrue as the other podcast works fine.

Just wondering if there's something I need to add in the template settings etc to make it work, I'm currently running the .mp3 file out using wireSendFile() and setting the type to application/mp3 in template settings but with no avail. I'm running nginx.

Any help would be appreciated.

Link to comment
Share on other sites

1 hour ago, benbyf said:

not sure how to do that in PW. But also it's mega strange it works on my v2.7 install but not on v3.

Not sure why it's working on 2.7 v 3, but setting headers in PW is just like any PHP script - just use the example in that SO thread and I am guessing it will work.

Link to comment
Share on other sites

If you have control over the server and you're able to install PECL module (pecl_http) on PHP, then http_send_file() function is the most straightforward way to implement partial downloads. Otherwise you have to implement it manually using a code similar to one in SO link above.

http://php.net/manual/fa/function.http-send-file.php

  • Like 1
Link to comment
Share on other sites

On 15/04/2017 at 6:36 PM, abdus said:

It looks like you need to set some headers to enable byte range requests and check incoming requests for byte ranges that client is currently at. 

Check out this SO answer:

http://stackoverflow.com/a/157447

That anwser did the trick i think, i managed to make the implementation below. It must have been a new itunes requirement for byte-range headers.

$filesize = filesize($page->mp3->httpUrl);
    $offset = 0;
    $length = $filesize;

    if ( isset($_SERVER['HTTP_RANGE']) ) {
        // if the HTTP_RANGE header is set we're dealing with partial content

        $partialContent = true;

        // find the requested range
        // this might be too simplistic, apparently the client can request
        // multiple ranges, which can become pretty complex, so ignore it for now
        preg_match('/bytes=(\d+)-(\d+)?/', $_SERVER['HTTP_RANGE'], $matches);

        $offset = intval($matches[1]);
        $length = intval($matches[2]) - $offset;

    } else {
        $partialContent = false;
    }

    $file = fopen($page->mp3->httpUrl, 'r');

    // seek to the requested offset, this is 0 if it's not a partial content request
    fseek($page->mp3->httpUrl, $offset);

    $data = fread($page->mp3->httpUrl, $length);

    fclose($page->mp3->httpUrl);

    if ( $partialContent ) {
        // output the right headers for partial content

        $headers = array(
            'HTTP/1.1 206 Partial Content' => true,
            'Content-Type' => 'audio/mp3',
            'Content-Length' => $filesize,
            'Content-Disposition' => 'attachment; filename="' . $fileName . '"',
            'Content-Range' => 'bytes ' . $offset . '-' . ($offset + $length) . '/' . $filesize,
            'Accept-Ranges' => 'bytes',
        );

        // send partial file
        wireSendFile($page->mp3->filename, $options, $headers);
    }else{
        // send file
        wireSendFile($page->mp3->filename, $options);
    }

 

  • Like 2
Link to comment
Share on other sites

@benbyf, I am not sure if this is the whole code, but I want to point out some parts.

<?php
// you should use $page->mp3->filename to prevent unnecessary (and slower) network request
// let PHP read file from the disk directly
$file = fopen($page->mp3->httpUrl, 'r');
fseek($page->mp3->httpUrl, $offset);
$data = fread($page->mp3->httpUrl, $length);
fclose($page->mp3->httpUrl);

// also, this forces client to download whole file
wireSendFile($page->mp3->filename, $options, $headers);

// whereas sending the parts you prepared is just cheaper
print($data);

So, the current implementation just sets the headers iTunes was checking for, but it sends whole file, not the parts client requested. This might total to an unnoticeable difference for lower traffic files, but you should opt for sending only the parts client requested.
Just my 2 cents.

  • Like 2
Link to comment
Share on other sites

Yeah, that might be confusing somewhat. To summarize the changes:

  • The part starting with $file = open($page->mp3->httpUrl, 'r'); should be
    <?php
    $filePath = $page->mp3->filename;
    
    $file = fopen($filePath, 'r');
    fseek($filePath, $offset);
    $data = fread($filePath, $length);
    fclose($filePath);
    to read parts of the file client requested directly from the disk.
  • While sending the file to the client, you should replace this part
    <?php
        // send partial file
        wireSendFile($page->mp3->filename, $options, $headers);
    } else {
        // send file
        wireSendFile($page->mp3->filename, $options);
    }

    with print($data); like so

    <?php
    // send partial file
        print($data);
    } else {
        // send file
        print($data);
    }

    Because you've set headers earlier, you dont need to set them again.

With these changes, hopefully, you'll have saved server from wasting some precious bandwidth.

Good luck :)

  • Like 4
Link to comment
Share on other sites

Are all podcasts in MP3 format? If 'Content-Type' => 'audio/mp3 header and mime type of podcasts do not match, you'd get the error. Try removing 'Content-Type' => 'audio/mp3' from $headers array.

In case it doesnt work, https://gist.github.com/benrolfe/5453074 claims to solve iTunes byte range problem. Changing the $file_path at line 2 to $page->mp3->filename should be enough.

  • Like 4
Link to comment
Share on other sites

  • 2 months later...
On 18.4.2017 at 6:39 PM, benbyf said:

yep that gist solves it. I just kept the wireSendFile() if the file isn't a partial content reponse and works well. Thanks @abdus!

new games podcast with my friends www.threepointspodcast.com/

@benbyf is it possible for your to share your current implementation for downloading/sending out the mp3? i tried to follow your steps but got confused with the last gist.

thanks!

Link to comment
Share on other sites

  • 5 months later...

Just checked on: http://castfeedvalidator.com/

and still not delivering partial headers, anyone have any experience on this, I'm running PW on nginx. Here is my current php code for the file request:

<?php namespace ProcessWire;
if($page->mp3){

    $page->of(false);
    $page->counter += 1;
    $page->save();
    $page->of(true);


    //
    // adapted from GIST: https://gist.github.com/codler/3906826
    //
    $options = array(
        // boolean: halt program execution after file send
        'exit' => true,
        // boolean|null: whether file should force download (null=let content-type header decide)
        'forceDownload' => false,
        // string: filename you want the download to show on the user's computer, or blank to use existing.
        'downloadFilename' => '',
    );

    $file_path = $page->mp3->filename;

    $fp = @fopen( $file_path, 'rb' );
    $size   = filesize( $file_path );
    $length = $size;
    $start  = 0;
    $end    = $size - 1;

    header("Content-type: audio/{$page->mp3->ext}");
    header( "Accept-Ranges: 0-$length" );
    header( "Content-Length: $length" );

    // find request headers for partial content
    if ( isset($_SERVER['HTTP_RANGE']) ) {
        // if the HTTP_RANGE header is set we're dealing with partial content
        $partialContent = true;

    } else {
        $partialContent = false;
    }

    if ( $partialContent ) {

        $c_start = $start;
        $c_end   = $end;

        // Extract the range string
        list(, $range) = explode( '=', $_SERVER['HTTP_RANGE'], 2 );
        // If the range starts with an '-' we start from the beginning
        // If not, we forward the file pointer
        if ( $range{0} == '-' )
        {
        	// The n-number of the last bytes is requested
        	$c_start = $size - substr($range, 1);
        }
        else
        {
        	$range  = explode( '-', $range );
        	$c_start = $range[0];
        	$c_end   = ( ( isset( $range[1] ) && is_numeric( $range[1] ) ) ? $range[1] : $size );
        };
        // End bytes can not be larger than $end.
        $c_end = ($c_end > $end) ? $end : $c_end;
        $start  = $c_start;
        $end    = $c_end;
        $length = $end - $start + 1;
        fseek( $fp, $start );

        // Start buffered download
        $buffer = 1024 * 8;
        while ( ! feof( $fp ) && ( $p = ftell($fp) ) <= $end )
        {
        	if ( $p + $buffer > $end )
        	{
        		$buffer = $end - $p + 1;
        	};

        	set_time_limit( 0 );

        	echo fread( $fp, $buffer );

        	flush();
        };
        header( "Content-Range: bytes $start-$end/$size" );
        header( 'HTTP/1.1 206 Partial Content' );

        fclose( $fp );

    }else{
        // send file
        wireSendFile($page->mp3->filename, $options);
    }
}

 

  • Like 1
Link to comment
Share on other sites

  • 2 months later...

Couldnt get the tracking and partial headers to work together so ive allowed the RSS feed direct link to the file and only track when viewing on the site. This seems to work now on itunes and means users can track through the audio which was erroring before :/

Link to comment
Share on other sites

  • 1 month later...
On 3/12/2018 at 11:21 AM, benbyf said:

Couldnt get the tracking and partial headers to work together so ive allowed the RSS feed direct link to the file and only track when viewing on the site. This seems to work now on itunes and means users can track through the audio which was erroring before ?

Here is some code I used which is working fine:

https://github.com/horst-n/LocalAudioFiles/blob/master/site-default/templates/local-audio-files_stream.php#L34-L45

https://github.com/horst-n/LocalAudioFiles/blob/master/site-default/templates/local-audio-files_stream.php#L264

 

  • Like 3
  • Thanks 1
Link to comment
Share on other sites

  • 1 month later...

Maybe a bit late, but I solved this with a Module:

https://gist.github.com/noelboss/7815ff2fea96b41544672f5e1a15f1f5

Usage:

wire('modules')->get('ProcessPageView')->sendFile($page, $filename);

I remember doing it that way, since I couldn't hook WireHttp or something and I needed to look up the file using the filename (protected site, using repeaters etc.) – you could also adapt it to accept a PageFile or PageImage or a path even… Maybe a bit more straight forward. 

Link to comment
Share on other sites

not sure if we have our wires crossed but providing the file wasnt really the issue, other wise I would just link to the location, it was capturing statistics on file access where range-headers needed to be provided. This was mainly as itunes at teh time didnt supply any podcast data – they've changed their tune abit since but still what they track is limited and to this day my podcast still says "Not Enough Data" sooooo still super annoying.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...