DvdFriend Site Update

Greetings

Site development has been on a hiatus for several weeks; some of it involuntary, and some of it just laziness. I had a bunch of side work to do for one client, and I built a prototype for another. (Lets see if that prototype turns into actual work).

Anyhoo, I've been able to work on DvdFriend quite a bit this week.

Pricing
- Pricing - PREVIOUSLY: I created the APIs to retrieve the data from the vendor sites and update the DvdFriend database. There's a EDIT PRODUCT page that uses that technology to do the initial parsing.

NOW: The next step is to update the prices automatically on a periodic basis. When the page loads, it checks when the price was last updated. If it was over a day, it needs to get fresh pricing. It needs to do it in the background, though. I don't want the page to hang for 5 or 10 seconds as it hits the vendors. Additionaly, if hundreds of people are hitting hundreds of pages, I don't want it to process all of those remote requests at once.

I spent more time on this then I needed to. I could've slapped together a quick solution to satisfy the immediate need. Instead, I created a ThreadQueue api that allows for great reuse for future needs. Basically, its just a queue. You stuff things on the queue. You then assign a delegate to the queue, and tell it how many can be processed at once. The object takes it from there. It will start up to x number of threads which process the queue. When each thread finishes with a work item, another thread is started that handles the next one. Neat. (It uses the thread pool, so the same threads are used repeatedly.)

Its a neat object. I spent the extra time on it so that I can make it distributable. I'll finish it up and make it available somewhere.

So anyway, when the page loads, it checks to see if the prices need to be refreshed. If so, it queues the items that need to be refreshed. The page shows the prices, but displays "refreshing" beneath them. A tooltip tells you what's going on, and suggests that you refresh in a few seconds. (Later, I'll wrap this up in an update panel and have it refresh automatically. 1 thing at a time.)

There's one problem, though. I only have about 50 prices in the DB. The original DB, which has all of the pricing, is sitting on the hard drive on my shut-down home computer. I get home tomorrow and leave again Saturday morning. I'll see if I can run the conversion somewhere in that window.

Alas, the moral of the story is that pricing is now FULLY functional. I just have to repopulate the data.

Admin Product Editor
The product page editor is where you assign vendor product ids to the product. Once I have the product ID, I can refresh the pricing whenever I need to. Currently, that's done just by dragging-and-dropping the url from one browser to the editor page. That's a pain, but there aren't many options. IE7 really locks down things like cross-frame scripting. I looked at creating a windows form program where that wouldn't be an issue, but the IE control constantlly crashes under XP PRO SP2. So, at the time I did what I could and just added a textbox. You paste the url, and it will parse out all of the information.

That's handy for updating products on a one-by-one basis. But, it needs a way to deal with things in mass. It needs to tie into lists of products that need pricing.

I created a view that assembles all products + all product types + all vendors and indicates if pricing exists for each one. It also specifies if any reviews or ratings exist for that product. From that view, I can now create child views that provide specific useful information.

The first one is "products with reviews and without pricing". Now I can run a quick query and see which products people have reviewed but don't have prices. I can display the list of the left side of the page and work through them one by one. (They'll probably be ordered by recent review date so that the most recent stuff can be updated).

The editor page has a button that launches a search in the vendor site. I use that as a starting point, then browse to the particular product I want. I then paste (or drag) the url to the parent page.

That's a step in the right direction, but its annoying.

Solution!

At last, I came up with a great solution. As I said, there are all types of security things that prevent me from doing this the way I used to do it. Finally, I worked out something that isn't blocked by security.

You can create an IE Favorite that has javacript in it.

IE: instead of http://www.dvdfriend.us (or anything else), you can create a favorite for
javascript:alert('hello');

Each time you click the favorite, it gives you an alert box.
I created this one

javascript:alert(document.location.href)

Once that worked, I was cautiously optimistic that I could solve my problems.

The next test was this:

javascript:document.location.href='http://mymachine/test.aspx?url=' + escape(document.location.href);

That javascript redirects to a specified page and passes the URL of the current page to it. So, the final URL is something like

http://mymachine/test.aspx?url=www.blogger.com
(That's a simplified version. The full url is actually escaped within the parameter. I omitted the special chars)

This is the big test. If security was going to interfere, this would be the place. It didn't!

Once that worked, I knew I as home free.

Here's how the pricing editor works:
- Go to the Edit Product page
- Select the item you want to get pricing for (IE: Transformers / HD-DVD)
- The button redirects to VendorProxy.aspx
The proxy page saves the product id and the product type id as cookies
It then redirects to the search results page for the vendor passing TRANSFORMERS as the search keyword
- I select the TRANSFORMERS HD DVD that I was looking for.
- I then click a favorite on my link bar. It takes the url of the current page and passes it to my VendorProxyReturn.aspx page.
- The VendorProxyReturn page:
- gets the product id and product type id out of the cookie
- gets the url out of the querystring
- uses those three pieces of information to get the pricing
- Redirects back to the edit product page. The pricing is now populate for that item.

I've been doing this pricing stuff since 1999 or 2000. I've had many solutions over the years, and some have been very good but required a lot of code that wasn't 100% reliable due to the intricacies of each vendor. This solution, while not the most robust, is the simplest and easiest.

From a admin user perspective:
- click the button which opens the search results page for that vendor
- find the product
- click the favorite link
Done!

Its really cool. I'm thrilled that I finally found something that works. Another key advantage of this is that its something I'm comfortable letting other admin users use.

Product Type Images
DvdFriend is very movie centric, obviously. But, it supports all types of products. Its currently setup XBOX games and books too. The problem is that the main page is dominated by movies. If I suddenly drop in a book review, how do you know it's a book review?

I added icons to indicate the product type. There's a journal type thing for a book, and everything else is a film reel. (I'll find a good game icon when I need it). I think it looks pretty good on the page.



These changes are all live on http://www.dvdfriend.us

What's Next?
I'd like to start putting in ratings for tv shows. I have to build a product hierarchy, which doesn't exist yet. When I built this version of the DB, I kept it very flat. Previous incarnations became to complicated due to the hierarchies. Flat is simple. I can use other objects to build a hierarchy as needed, such as with TV shows, but not force it where it's not needed.