Jump to content
benbyf

byte-range requests for podcast mp3

Recommended Posts

related to this topic on wireSendFile(). I've been trying to create a RSS feed for a podcast which i've done previously using PW 2.7.2 with no trouble (e.g. http://machine-ethics.net/), however sitting on the same infustructure a new site (http://www.threepointspodcast.com/) on PW 3.0.42 is having issues validating with itunes with this error from them: Can’t submit your feed. Your episodes are hosted on a server which does not support byte-range requests. Enable byte-range requests and try again. Which is obviously untrue as the other podcast works fine.

Just wondering if there's something I need to add in the template settings etc to make it work, I'm currently running the .mp3 file out using wireSendFile() and setting the type to application/mp3 in template settings but with no avail. I'm running nginx.

Any help would be appreciated.

Share this post


Link to post
Share on other sites

It looks like you need to set some headers to enable byte range requests and check incoming requests for byte ranges that client is currently at. 

Check out this SO answer:

http://stackoverflow.com/a/157447

  • Like 4

Share this post


Link to post
Share on other sites

not sure how to do that in PW. But also it's mega strange it works on my v2.7 install but not on v3.

Share this post


Link to post
Share on other sites
1 hour ago, benbyf said:

not sure how to do that in PW. But also it's mega strange it works on my v2.7 install but not on v3.

Not sure why it's working on 2.7 v 3, but setting headers in PW is just like any PHP script - just use the example in that SO thread and I am guessing it will work.

Share this post


Link to post
Share on other sites

If you have control over the server and you're able to install PECL module (pecl_http) on PHP, then http_send_file() function is the most straightforward way to implement partial downloads. Otherwise you have to implement it manually using a code similar to one in SO link above.

http://php.net/manual/fa/function.http-send-file.php

  • Like 1

Share this post


Link to post
Share on other sites

im still puzzled why it works on the same infustructure but different versions of PW.

Share this post


Link to post
Share on other sites

@benbyf

just a shot in the dark, but have you considered that with the 2.7.2 version, when you submitted it, Apple was not validating the byte-range request support?

  • Like 2

Share this post


Link to post
Share on other sites
21 hours ago, Macrura said:

@benbyf

just a shot in the dark, but have you considered that with the 2.7.2 version, when you submitted it, Apple was not validating the byte-range request support?

could be the case.

Share this post


Link to post
Share on other sites
On 15/04/2017 at 6:36 PM, abdus said:

It looks like you need to set some headers to enable byte range requests and check incoming requests for byte ranges that client is currently at. 

Check out this SO answer:

http://stackoverflow.com/a/157447

That anwser did the trick i think, i managed to make the implementation below. It must have been a new itunes requirement for byte-range headers.

$filesize = filesize($page->mp3->httpUrl);
    $offset = 0;
    $length = $filesize;

    if ( isset($_SERVER['HTTP_RANGE']) ) {
        // if the HTTP_RANGE header is set we're dealing with partial content

        $partialContent = true;

        // find the requested range
        // this might be too simplistic, apparently the client can request
        // multiple ranges, which can become pretty complex, so ignore it for now
        preg_match('/bytes=(\d+)-(\d+)?/', $_SERVER['HTTP_RANGE'], $matches);

        $offset = intval($matches[1]);
        $length = intval($matches[2]) - $offset;

    } else {
        $partialContent = false;
    }

    $file = fopen($page->mp3->httpUrl, 'r');

    // seek to the requested offset, this is 0 if it's not a partial content request
    fseek($page->mp3->httpUrl, $offset);

    $data = fread($page->mp3->httpUrl, $length);

    fclose($page->mp3->httpUrl);

    if ( $partialContent ) {
        // output the right headers for partial content

        $headers = array(
            'HTTP/1.1 206 Partial Content' => true,
            'Content-Type' => 'audio/mp3',
            'Content-Length' => $filesize,
            'Content-Disposition' => 'attachment; filename="' . $fileName . '"',
            'Content-Range' => 'bytes ' . $offset . '-' . ($offset + $length) . '/' . $filesize,
            'Accept-Ranges' => 'bytes',
        );

        // send partial file
        wireSendFile($page->mp3->filename, $options, $headers);
    }else{
        // send file
        wireSendFile($page->mp3->filename, $options);
    }

 

  • Like 2

Share this post


Link to post
Share on other sites

@benbyf, I am not sure if this is the whole code, but I want to point out some parts.

<?php
// you should use $page->mp3->filename to prevent unnecessary (and slower) network request
// let PHP read file from the disk directly
$file = fopen($page->mp3->httpUrl, 'r');
fseek($page->mp3->httpUrl, $offset);
$data = fread($page->mp3->httpUrl, $length);
fclose($page->mp3->httpUrl);

// also, this forces client to download whole file
wireSendFile($page->mp3->filename, $options, $headers);

// whereas sending the parts you prepared is just cheaper
print($data);

So, the current implementation just sets the headers iTunes was checking for, but it sends whole file, not the parts client requested. This might total to an unnoticeable difference for lower traffic files, but you should opt for sending only the parts client requested.
Just my 2 cents.

  • Like 2

Share this post


Link to post
Share on other sites

cooooooooool, will try and amend. Your 2 cents is super useful but seems your code was a mix of your suggestions and code needing amending no? bit confusing sorry

Share this post


Link to post
Share on other sites

Yeah, that might be confusing somewhat. To summarize the changes:

  • The part starting with $file = open($page->mp3->httpUrl, 'r'); should be
    <?php
    $filePath = $page->mp3->filename;
    
    $file = fopen($filePath, 'r');
    fseek($filePath, $offset);
    $data = fread($filePath, $length);
    fclose($filePath);
    to read parts of the file client requested directly from the disk.
  • While sending the file to the client, you should replace this part
    <?php
        // send partial file
        wireSendFile($page->mp3->filename, $options, $headers);
    } else {
        // send file
        wireSendFile($page->mp3->filename, $options);
    }

    with print($data); like so

    <?php
    // send partial file
        print($data);
    } else {
        // send file
        print($data);
    }

    Because you've set headers earlier, you dont need to set them again.

With these changes, hopefully, you'll have saved server from wasting some precious bandwidth.

Good luck :)

  • Like 4

Share this post


Link to post
Share on other sites

think maybe my Headers are still wrong as it has a problem when seeking - gives me a "video or MIME type not supported" error

Share this post


Link to post
Share on other sites

Are all podcasts in MP3 format? If 'Content-Type' => 'audio/mp3 header and mime type of podcasts do not match, you'd get the error. Try removing 'Content-Type' => 'audio/mp3' from $headers array.

In case it doesnt work, https://gist.github.com/benrolfe/5453074 claims to solve iTunes byte range problem. Changing the $file_path at line 2 to $page->mp3->filename should be enough.

  • Like 4

Share this post


Link to post
Share on other sites
On 18.4.2017 at 6:39 PM, benbyf said:

yep that gist solves it. I just kept the wireSendFile() if the file isn't a partial content reponse and works well. Thanks @abdus!

new games podcast with my friends www.threepointspodcast.com/

@benbyf is it possible for your to share your current implementation for downloading/sending out the mp3? i tried to follow your steps but got confused with the last gist.

thanks!

Share this post


Link to post
Share on other sites

Just checked on: http://castfeedvalidator.com/

and still not delivering partial headers, anyone have any experience on this, I'm running PW on nginx. Here is my current php code for the file request:

<?php namespace ProcessWire;
if($page->mp3){

    $page->of(false);
    $page->counter += 1;
    $page->save();
    $page->of(true);


    //
    // adapted from GIST: https://gist.github.com/codler/3906826
    //
    $options = array(
        // boolean: halt program execution after file send
        'exit' => true,
        // boolean|null: whether file should force download (null=let content-type header decide)
        'forceDownload' => false,
        // string: filename you want the download to show on the user's computer, or blank to use existing.
        'downloadFilename' => '',
    );

    $file_path = $page->mp3->filename;

    $fp = @fopen( $file_path, 'rb' );
    $size   = filesize( $file_path );
    $length = $size;
    $start  = 0;
    $end    = $size - 1;

    header("Content-type: audio/{$page->mp3->ext}");
    header( "Accept-Ranges: 0-$length" );
    header( "Content-Length: $length" );

    // find request headers for partial content
    if ( isset($_SERVER['HTTP_RANGE']) ) {
        // if the HTTP_RANGE header is set we're dealing with partial content
        $partialContent = true;

    } else {
        $partialContent = false;
    }

    if ( $partialContent ) {

        $c_start = $start;
        $c_end   = $end;

        // Extract the range string
        list(, $range) = explode( '=', $_SERVER['HTTP_RANGE'], 2 );
        // If the range starts with an '-' we start from the beginning
        // If not, we forward the file pointer
        if ( $range{0} == '-' )
        {
        	// The n-number of the last bytes is requested
        	$c_start = $size - substr($range, 1);
        }
        else
        {
        	$range  = explode( '-', $range );
        	$c_start = $range[0];
        	$c_end   = ( ( isset( $range[1] ) && is_numeric( $range[1] ) ) ? $range[1] : $size );
        };
        // End bytes can not be larger than $end.
        $c_end = ($c_end > $end) ? $end : $c_end;
        $start  = $c_start;
        $end    = $c_end;
        $length = $end - $start + 1;
        fseek( $fp, $start );

        // Start buffered download
        $buffer = 1024 * 8;
        while ( ! feof( $fp ) && ( $p = ftell($fp) ) <= $end )
        {
        	if ( $p + $buffer > $end )
        	{
        		$buffer = $end - $p + 1;
        	};

        	set_time_limit( 0 );

        	echo fread( $fp, $buffer );

        	flush();
        };
        header( "Content-Range: bytes $start-$end/$size" );
        header( 'HTTP/1.1 206 Partial Content' );

        fclose( $fp );

    }else{
        // send file
        wireSendFile($page->mp3->filename, $options);
    }
}

 

  • Like 1

Share this post


Link to post
Share on other sites

Couldnt get the tracking and partial headers to work together so ive allowed the RSS feed direct link to the file and only track when viewing on the site. This seems to work now on itunes and means users can track through the audio which was erroring before :/

Share this post


Link to post
Share on other sites

Just had a need to use this profile.  Thanks for your work on this profile.

  • Like 1

Share this post


Link to post
Share on other sites
On 3/12/2018 at 11:21 AM, benbyf said:

Couldnt get the tracking and partial headers to work together so ive allowed the RSS feed direct link to the file and only track when viewing on the site. This seems to work now on itunes and means users can track through the audio which was erroring before 😕

Here is some code I used which is working fine:

https://github.com/horst-n/LocalAudioFiles/blob/master/site-default/templates/local-audio-files_stream.php#L34-L45

https://github.com/horst-n/LocalAudioFiles/blob/master/site-default/templates/local-audio-files_stream.php#L264

 

  • Like 3
  • Thanks 1

Share this post


Link to post
Share on other sites

Maybe a bit late, but I solved this with a Module:

https://gist.github.com/noelboss/7815ff2fea96b41544672f5e1a15f1f5

Usage:

wire('modules')->get('ProcessPageView')->sendFile($page, $filename);

I remember doing it that way, since I couldn't hook WireHttp or something and I needed to look up the file using the filename (protected site, using repeaters etc.) – you could also adapt it to accept a PageFile or PageImage or a path even… Maybe a bit more straight forward. 

Share this post


Link to post
Share on other sites

not sure if we have our wires crossed but providing the file wasnt really the issue, other wise I would just link to the location, it was capturing statistics on file access where range-headers needed to be provided. This was mainly as itunes at teh time didnt supply any podcast data – they've changed their tune abit since but still what they track is limited and to this day my podcast still says "Not Enough Data" sooooo still super annoying.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By clsource
      Hello I made a simple app for reading the main RSS for ProcessWire news.
      Now you can access ProcessWire Blog, ProcessWire weekly and the Latest Forum Posts in a Single App
      in your iOS or Android smartphone. Open Source of Course.
      Made using the http://jasonette.com technology.

      You can compile your own app if you want.
      For A Quick Look

      1.- Download the Jason App

      (iOS) https://itunes.apple.com/us/app/jason./id1095557868?mt=8
      (Android) https://play.google.com/store/apps/details?id=com.jasonette.jason

      2.- Use the Following Url

      https://raw.githubusercontent.com/NinjasCL/pw-app/master/app.json

       

       
      Source Here
      https://github.com/NinjasCL/pw-app
       
       
    • By muffin-man
      I just recently found about processwire a few days ago and now I am currently stuck on modules. I followed the directions on installing it and then inserted the example codes into both a blank template and a pre-built template I made and it shows nothing. I also changed the rss link to something from CNN to see if it was just the URL, but it still shows nothing. I'm not sure if it has something to do with the get function. I implemented this sample into a php template.
      <?php $rss = $modules->get("MarkupLoadRSS");  $rss->load("http://www.cnn.com/services/rss/"); foreach($rss as $item) {  echo "<p>"; echo "<a href='{$item->url}'>{$item->title}</a> "; echo $item->date . "<br /> "; echo $item->description;  echo "</p>"; } ?>  
      Please advise
    • By benbyf
      HELLO!
      Just in case anyone's interested I've just started this podcast http://www.machine-ethics.net/
      I sent less than a day building the site so be kind. I'm looking to improve it and make it available as a site profile as it has an rss feed that itunes can pick up and use which may be useful for anyone else creating podcasts.
      cheers
    • By aren
      Hi everyone!
      I've been away from web dev for a while (designing more), so I'm feeling a bit out of touch.
      Can anyone tell me what's the best way to combine multiple rss feeds in one single site/page these days? Preferably with javascript (or is there a better way these days?)
      What I want to do is to fetch only the images of multiple (hundreds, probably) rss feeds and display them in one site/page as one big gallery.
      Thanks, and cheers! 
    • By melissa_boyle
      Hi Guys,
      I have recently joined bloglovin and have implemented the RSS enhanced module onto my blog, encolsures for images are working ok but I was wondering if I can implement these to be wrapped in an image tag to allow blog lovin to be able to carry imagery through.
      Any help would be greatly appreciated.
      Thanks,
      Mel
×
×
  • Create New...