A have been checking out a recent application called Scrapblog and looking under the hood at how the put the application together. I know they use XML to markup the coordinates and properties of media elements on the page.  What surprised me was that they take a Bitmap image of each slide and save that as a single JPEG file when you publish your Scrapblog.   I had just assumed they copy images from Flickr to the server and referenced those images in the published content.   I even thought they might dynamically link the images from Flickr.  I did notice though, that Scrapblog links to Youtube videos dynamically. 

They are obviously not using the Youtube API because Youtube blew that up by making a security policy change to the crossdomain.xml file needed to pull videos into Flex.   So I did a little investigation.  I found a site, in French, that had a Flash 8 application to basically screen scrape and decode the URL to Youtube videos.   I migrated this over to Flex and created a small demo application.

Here is the code for the application:

<?xml version="1.0" encoding="utf-8"?>
<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
               xmlns:s="library://ns.adobe.com/flex/spark"
               xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="955" minHeight="600">

    <fx:Script>
        <![CDATA[
            import mx.rpc.events.FaultEvent;
            import mx.rpc.events.ResultEvent;

            private var page:XML;
            private var _src:String;

            [Bindable]
            private var youtube:String;


            private function getYoutube(event:Event):void
            {
                _src = youtubeURL.text;
                scrape.text = "";
                youtubeService.url = _src;
                youtubeService.send();
            }

            private function getVideoID(src:String):String
            {
                var id:String = src.substr(src.indexOf("v=")+2);
                var nextVarPos:Number = id.indexOf("&");
                if(nextVarPos == -1)
                    return id;
                return id.substr(0,nextVarPos);
            }

            private function onResult(event:ResultEvent):void
            {
                //scrape.text = "\n" + scrape.text + "Screen result: \n\n" + event.result + "\n";

                var xml:String = event.result as String;
                var index1:Number = xml.indexOf('var swfConfig =');
                var index2:Number =  xml.indexOf('video_id', index2);

                var str:String = xml.substr(index1,  index2)

                var t:String = str.substr(str.indexOf("\"t\": ")+5);
                t = t.substr(0, t.indexOf(",") );

                if(t != "" && t.length > 0) {

                    youtube = "http://youtube.com/get_video.php?video_id=" + getVideoID(_src) + "&t=" + t;

                    scrape.text = "Youtube URL: " + youtube;
                    youtubePlayer.source = youtube ;
                    youtubePlayer.play();
                } else {
                    scrape.text = "There was an error parsing the contents of the page, we could not find a SWFObject.";
                }
            }


            private function onFault(event:FaultEvent):void
            {
                scrape.text = " \n Error: " + event.fault;
            }
        ]]>
    </fx:Script>

    <fx:Declarations>
        <s:HTTPService id="youtubeService" resultFormat="text"
                       fault="onFault(event)" method="GET"
                       result="onResult(event)"/>
    </fx:Declarations>


    <s:TextInput id="youtubeURL" text="http://www.youtube.com/watch?v=IpVWZePn1Y8" x="133" y="74" width="375"/>
    <s:Label x="42" y="76" text="Youtube URL:" fontWeight="bold"/>
    <s:Label x="33" y="129" text="Youtube Video:" fontWeight="bold"/>
    <s:Label x="42" y="421" text="Screen Scrape:" fontWeight="bold"/>
    <s:Button x="516" y="74" label="Get" click="getYoutube(event)"/>
    <s:VideoDisplay  id="youtubePlayer"  x="133" y="129" width="320" height="240"/>
    <s:Label x="42" y="10" text="Visit the page that contains the video, copy the URL on the right side of the page, and paste it in the field below." width="520" height="56" fontWeight="bold" fontSize="14"/>
    <s:Label x="53" y="377" text="* Be patient as screen scrapping a site is not always a fast process.  It may even blow up!"/>
    <s:TextArea x="133" y="420" width="429" height="146" id="scrape"/>

</s:Application>

To use it, just grab the URL for a Youtube video and past in the application (i.e. http://www.youtube.com/watch?v=IpVWZePn1Y8). The application actually parses the URL and takes the last bit after the “v=” to construct another URL to the actual FLV file.

I don’t think this is the most practical approach to grabbing Youtube videos and I am not entirely sure this is the only method, I wanted to put the application out there for comments and suggestions.   Another suggested solution was to use the BrightCove API.  Here is my updated attempt at the grabber using screen scraping:

Youtube Grabber (right-click to see source)

Update 7/17/2010

I found an excellent blog post similar to mine, it was posted sooner and works a bit better. Check out YouTube FLV URL post by Abdul. This one works great for getting the URL to Youtube videos.

I have also updated the application to work with Flex 4 and fixed an issue with the screen scrapper. Screen scraping isn’t the best approach because if Google changes the embed code (like they did recently), your screen scrapper will break and have to be updated. Plan on updating the application often as Google makes changes to the way they embed FLV files.

However because of security restrictions, the example will not run in the browser without a proxy. It will run from the Flex IDE and from an AIR application that can by-pass the Flash security sandbox.