Monday, January 20, 2014

For Our Dogster Friends

Update February 11th: Monster Speedup and Experimental Photobucket Rules

One thing that has become quickly apparent is that HTTrack can be ridiculously slow especially on bigger pages.   I have found a way to speed it up.  Way up. It appears that it throttles the download speed without telling you. There is a spot in the user interface that lets you set the maximum speed, but that does nothing unless you know a trick

Here is a shortcut to getting the speedup. Try putting the following in your Scan Rules
--disable-security-limits
--max-rate 1500000
--assume php=text/html
The first one is the secret to taking the brakes off. The second one is the maximum rate in bits per second. We have a two megabit download rate at our house, so I've set the maximum to three quarters of that, or 1.5 megabits per second. If you set this number too big, it just means it will hog your entire network connection. A rough rule of thumb might be that if you are getting internet through your phone company, try 1500000 as we have. If to the cable company, they are usually faster so try 5000000 (five million).

The "assume" bit is recommended in the HTTrack manual to speed up how it parses the files if you can make certain assumptions, and this assumption seems to work great.  Now it is only taking minutes to download smaller pages. The difference is night and day.  These scan rules are now in the main body of the blog post.

Not in the main body of the blog post is some new filters I have been using that seems to work well for downloading from Photobucket.
+*.photobucket.com/*.jpg
+*.photobucket.com/*.gif
+*.photobucket.com/*.png
-blog.photobucket.com/*
-secure.photobucket.com/*
-support.photobucket.com/*
-tutorials.photobucket.com/*
The lower in the list, the higher the priority, so the order above of these filters is important. The rules start by saying you can only load .jpg, .gif, and .png files from Photobucket. That stops it from downloading all the html pages that then takes you down the rabbit hole and downloading the entire site. After that, it specifically blacklists (with the minus signs in front) the blog, secure, support, and tutorial sections of the site. This is just to be on the safe side.  This has worked well for all the sites we have tried, but keep an eye on it if you decide to try it.  I have not put this in the main body of the blog post but give it a shot.

Finally, another problem we've been having is that some pictures weren't showing up after a successful download. I suspect this is because of occasional heavy traffic clobbering the site. Try the following if you are having trouble:
  • From the same window the Scan Rules are entered in, select the "Flow Control" tab
  • In that tab, enter 5 into the Retries field. 
  • Everything else is as per usual. 
Good luck!

Update February 9th: Disable Photobucket Scan Rules

It sounds like some people are running issues where HTTrack wants to download half of Photobucket.  It looks like you will need to figure out more specific scan rules than the ones I have in there now.  I have removed references to Photobucket from the post.

Update February 8th: Fixes and the Addition of Catster!

I have updated the scan rules to fix a few things and have created a set of scan rules for Catster as well.  The new rules are in the main body of the post.  A shout-out to the Dogster (and Catster!) folks that have helped out so much.
  • The gift archive now works.  A new scan filter "+www.dogster.com/subscr/gift_archive.php?pet_id=590841" (or whatever your pet's number is) does the trick.
  • Thumbnails in photo comments now work.  Dogster, for some bizarre reason, has those thumbnails in a URL that looks like "+files.dogster.com./blahblahblah".  Did you catch the weird period between "com" and "/" in that last sentence?  Neither did I at first.  Tweaking the Scan Rule from "+files.dogster.com/*" to "+files.dogster.com*" did the trick.
  • Added a scan rule"+badge.dogster.com/*" to fix missing badges.
  • Added a few new scan rules that should help images from flickr, photobucket, and one other site I'd never heard of come through: "+*mytextgraphics.com" "+*.photobucket.com/*",  "+farm*.staticflickr.com/*", and "+farm*.static.flickr.com/*".  You can skip these if you don't have images off the main site, but leaving them in doesn't hurt anything.
  • Added a set of scan rules for Catster.  Did you know that the images on Catster are stored on servers named Dogster?  Neither did I at first.  For this reason, the Catster list will have a "+files.dogster.com*" entry in it on purpose.  It will also have a "+files.catster.com*" on it that I doubt is needed, but better safe than sorry.
  • Added a scan rule that pulls in a specific file used to format the display of the photo pages "+www.dogster.com/photos/photo.css", though I didn't really see a difference when it was added.
  • Reordered the list of scan rules to be a little more sane
Attempts by the Dogster folks and myself to display images in the Pup Pals pages have failed miserably.  I suspect that it has something to do with that being a page generated on the fly with PHP and this is a known issue with HTTrack, but the gift history is also PHP and it works.  Weird.

It doesn't hurt to test the archive out by disconnecting your Internet connection, clearing your browsers cache (Google is your friend here), and then bringing up the web page created by HTTrack for your pet to see if anything is unexpectedly missing.  Note that the page might be slow to load this way because it is trying to reach out to some sites like Facebook, Google, and Twitter that I have intentionally not mirrored.  Your browser will wait for a bit, give up, and then load the archive anyway.  These links don't affect the display of your stuff and they'll be there long after Dogster Doomsday.  Once you connect the Internet back up, it will reach these sites normally anyway

If you are an old hand at this archive process, the only substantial changes to this post are in the "Specify The Options" part.  The rest is the same, so you can save yourself some reading.
Finally, some of you might be wondering why this archive takes so damn long sometimes.  I was mystified by this for a while too until I ran a test on a cat that had 36 pages (!) of diary entries.  I checked out the folder where the diaries are stored and there were a ton more entries than that.

WHY?

Well, there might be 36 pages of diary entries, but there are also five or six diary entries per page.  If you click on a "Comments" link, a whole new page opens up for just that diary entry.  Same thing for photos.  So if you have ten pages of diary entries and ten pages of photos, you don't end up downloading twenty pages.  You download twenty pages where entries are grouped together PLUS over a hundred pages dedicated to each individual diary entry and photo.  Yeah, that is a lot, but that is the only way to see the comments.  It also doesn't help that the download speeds from Dogster and Catster seems to be less than speedy in the first place.

"Beepers" on the Dogster forum posted a suggestion on how to speed up an update if you already have an existing download.
  • On the first option page, you can choose "Existing project name".  Click the arrow to choose the one you want to update.
  • On the next page/screen, in the "Action" block, click the arrow again and choose *Update existing download.
  • Then click on "Set options" and the "Scan Rules" tab as usual.  All your rules for the previous save will be there.  Just add the new rules at the bottom and proceed as usual.
Give it a shot!
          

Update February 5th:

The Bad News

Folks on the Dogster forums groups discovered that the archives would not work when they had no Internet connection.  That implied that the download would stop working once Dogster shut down.  I verified that this is a problem and am very sorry to say that all the downloads that everyone has done to this point will be useless once Dogster shuts down. The web browser was doing some stuff behind the scenes that I wasn't aware of.

The Good News

I have a fix that I would like people to try. I have tested this on a couple of pages now with my browser cache cleared and our internet disconnected and it seems to work perfectly. It still has the same limitations as before (can't click on gifts to get details, etc) but all the important stuff will be there. You will still be able to navigate around the page just like you did before.

The Fix

A new set of Scan Rules is now in the main body of the post.  The rest of the steps should be the same.  Try these new scan rules out and post feedback in the comments.  If you are computer savvy, you can double check by clearing your browser cache, disconnecting your Internet connection, and loading the archived page.  If everything looks good, you are ready for Doomsday when Dogster shuts down.

Please test this out and give feedback in the comments.  Once again, I'm sorry for my error. If I knew what I was doing here, it would help!

P.S.

Some people have been worried about seeing stuff like this in their browser address bar.

file:///C:/My%20Web%20Sites/cocopuffondogster/www.dogster.com/dogs/500107.html

The worry is that www.dogster.com is showing up there still.  That isn't actually a problem as long as the start of the address starts with "file:///C:/blahblahblah".  In this case the "www.dogster.com" part in the middle is actually now a directory on your hard drive.  You can use Windows Explorer or whatever to actually navigate to that directory and below.  There is all kinds of neat stuff tucked away in there!

          

Update Jan 25th:

See near the bottom of the page for instructions on how to save your videos!  A couple limitations of this technique are also discussed.  And after all of this, look who was made Dog of The Day for January 25th, 2014!!!


          

One of the saddest times of my life was when our dear dog Abby passed. She was my best friend and a big part of our lives. That was about a year and a half ago now and it still brings a lump to my throat just thinking about that awful time.

In the days that followed, I was struck by the sympathy and compassion expressed by My Lovely Wife’s friends in the Dogster community. Dogster is a website that dog lovers from all over the world congregate to share photos, stories, and advice on their best friends. While I don't doubt that some of these folks might be a few Milk Bones short of a full bag, My Lovely Wife has developed a true friendship with many people on that site, none of whom she has ever met in person. Our dog now gets more Christmas cards than we do.

Unfortunately, word went out a short time ago that Dogster will be closing down. All of the pet pages with the photos, diaries, and associated comments will disappear. People are understandably upset because they don't know how to save all of those memories. Some are printing the pages off, some are resorting to screenshots.  And some are just at a loss.

But I think I know a better way. And I'll show you how to do it yourself. Step by step.  When you are done, you'll have all of your stuff for your dog.  It won't copy forum discussions and stuff like that.  But it will copy all of your diary entries, all of your photos, and all of the comments on that stuff.  And it will store all that away on your local hard drive.

Ready to go?  Good.  Start with...

Read This Whole Thing Through

Like a recipe, I highly recommend reading this through once or twice to get some idea of what you'll be doing.  It does assume you know your way around your computer a little bit, but you don't need to be a guru or anything like that.  If you are comfortable installing a program on your computer and navigating around the various files on your PC, you should be just fine.

Download HTTRack

HTTrack bills itself as a website copier.  Click on this link, download the appropriate file for your PC, and install it on your computer.  There are a bunch of different versions for the various flavors Windows and whether you are running the 32-bit or 64-bit version, so choose wisely.  You'll be out of luck if you are on a Mac (sorry) unless you know how to compile your own programs.  There looks to be versions for various Linux distributions though for the 0.001% of Dogster users on that operating system.

During the installation process, the program will ask you if you want to put an icon on the Desktop and in the Start Menu.  Let it do so.

Start WinHTTrack

Click the icon the installer put on your Desktop.  The program should start up, displaying one of the ugliest user interfaces in existence.  That's OK.  We aren't using it for its looks.  When you get the screen below, hit the "Next >" button to continue.


Start a New Project

We start out by defining a new project as follows:
  • New project name: This is what the program will name the directory where it stores the files you will download.  I'll be using Abby's Dogster page as an example, so I called my project "AbbyOnDogster".  Use whatever name you like here.
  • Project category: Just leave this blank
  • Base path: If you were going to download multiple Dogster pages, then all of them would go into this directory.  You can use the default "C:\My Web Sites" (it works fine) or specify any other folder you like.

Once you're entered that information, hit the "Next >" button to continue.

Specify the Site To Mirror

So far so good, but now it starts getting a little trickier.  Fill out the next window as follows.
  • Action: Stick with the default of "Download web site(s)"
  • Web Addresses: (URL): The best way to fill this in is to navigate to the Dogster page you want to clone, select the address area, copy it (Control-C), and paste it (Control-V).  Or just type it in from scratch.  The window below shows the address for Abby's page.  It will look something like http://www.dogster.com/dogs/590841.  Click that link to see what I mean.
  • The tricky part is coming up.  Do not hit the "Next >" button yet.  Instead, select the "Set options..." button above it.

Specify the Options

(This Section updated February 5th and then on February 8th to fix the Scan Rules as mentioned at the top of this post).

When you hit the "Set options...." button, there will be a bunch of tabs on top.  Click on the tab that says "Scan Rules".  Don't worry about any of the other tabs.  See the area I have circled in blue?  Click your mouse in that area and delete every single character in there.  You want it to be perfectly empty.  Do not hit the "Exclude link(s)..." button.  Do not hit the "Include link(s)..." button.


Now, copy the four thirteen all of the lines of text below and paste it into the now empty area that has the blue circle above.  Cut and paste is your friend here.
-*
+www.dogster.com/dogs/590841
+www.dogster.com/dogs/590841/*
+www.dogster.com/subscr/gift_archive.php?pet_id=590841
+www.dogster.com/static/*
+www.dogster.com/files*
+www.dogster.com/images/*
+badge.dogster.com/*
+files.dogster.com*
+www.dogster.com/photos/photo.css
+*.cdnsters.com/*
+tetra.saymedia.com/*
+*amazonaws.com/*
+use.typekit.com/*
+*mytextgraphics.com*
+farm*.staticflickr.com/*
+farm*.static.flickr.com/*
--disable-security-limits
--max-rate 1500000
--assume php=text/html
Note that there is now a * after "+files.dogster.com".  This is a change from my first and second cracks at this.  Don't miss that one.

What if you have a cat?  Other than having to vacuum hairballs off the carpet on a regular basis, you want to use this set of scan rules instead.  Abby was not a cat, but work with me here.  Again, cut and paste is your friend.
-*
+www.catster.com/cats/590841
+www.catster.com/cats/590841/*
+www.catster.com/subscr/gift_archive.php?pet_id=590841
+www.catster.com/static/*
+www.catster.com/files*
+www.dogster.com/files*
+www.catster.com/images/*
+badge.catster.com/*
+files.catster.com*
+files.dogster.com*
+www.catster.com/photos/photo.css
+*.cdnsters.com/*
+tetra.saymedia.com/*
+*amazonaws.com/*
+use.typekit.com/*
+*mytextgraphics.com*
+farm*.staticflickr.com/*
+farm*.static.flickr.com/*
--disable-security-limits
--max-rate 1500000
--assume php=text/html
As explained up top, there are lines for both "+files.catster.com*" and "+files.dogster.com*" because Catster's images are actually stored on Dogster's servers.

This might look like Greek to you, but every single character here is important.  Make sure that these four thirteen lines get copied EXACTLY into the area circled in blue above.  That first line is indeed a minus sign followed by an asterisk.  All the other lines start with a plus.  All of this is important.

Or almost all of it.  The "+*mytextgraphics.com/*" won't be necessary for most people and could be safely left out.  One of our friends had a link to this website for an animated image of her pets name so this had to be included to get that to show up.  Others had images from Flickr (+farm*.staticflickr.com/*) and Photobucket (+*.photobucket.com/*).  Again, you can leave these out if your page doesn't link to these sites, but leaving them in there doesn't hurt anything either.  You would have to do something similar for any other off-Dogster sites that you link to for special images like that.  Just be sure to start the name with a plus and follow the format I've used.

You should end up with something that looks exactly like this when you are done.  Unfortunately the Scan Rules list is kind of long now so stuff is getting cut off on the bottom of the list.  But you get my drift.

Now if we were to proceed on from this point, it would copy Abby's page on Dogster because her Dogster ID number is 590841.  What you need to do now is replace the number 590841 with your own dog's ID number in all three places: the second, third, and fourth lines of the list.  Your dog's ID number is the one that shows up in the address bar of your web browser whenever you go to your dog's pet page.  It is usually six or seven digits long.  Do not add or subtract asterisks, slashes, or plus signs, or anything else unless you really know what you are doing.  Just change the number.  Get this wrong and you could end up downloading half of YouTube.  Ask me how I know...

Once you have changed 590841 to your own dog's ID number, hit the "OK" button at the bottom of the window and pat yourself on the back.  The hard part is done.

After hitting the OK button in the window above, you'll be back to the window that offered you the "Set Options" button in the first place.  You've already done that, so you can hit the "Next" button to Continue.


Breeze Past Connection Parameters

The next screen you'll see is shown below.  This is just getting fancy now.  You shouldn't have to worry about anything on this window, so hit the "Finish" button at the bottom.  And cross your fingers, because once you hit "Finish" the download will begin.


Watch the Download

Here is where it gets exciting.  The program is now going to go out and fetch your Dogster page, with all the associated diary entries, photos, and comments.  It is going to store them on your hard drive and change all the links around so that everything works on your local computer.  What it won't do is  create a copy of all the forum discussions or download all of your friends pages.  It gets your stuff for your dog.  And that is the most important thing.


How long this part takes depends on how much stuff you put on Dogster in the first place and the speed of your Internet connection.  Our connection isn't that great but there is a fair bit of stuff on Abby's page, and it still took around five fifteen minutes (the bigger set of Scan Rules from my first crack at this causes a fair bit more stuff to be downloaded).  As it runs, you'll hopefully see some familiar text going by.  The window above has some text that says "...bys_adventures", short for "Abby's Adventures".  This kind of thing told me I was on the right track.

Success!

When your pet's page has been copied, you should see a screen like this. Click "Browse Mirrored Website" and hopefully what you'll be looking at is a copy of your pet's Dogster page.

 

The actual page will be found where you specified it in that very first screen.  In this example, it would be in a place called "C:\My Web Sites\AbbyOnDogster".  That file will have a directory in it called "index.html".  Just click on that anytime you like and you should see a replica of your dog's pet page.  For our example, the address bar should look something like this.


See how it says "file:///C:/My Web Sites...".  That means that this file is on my local hard drive.  Contrast this with what it says when I access it over the web:

If everything went well, you'll have a copy of your dog's pet page tucked away nice and safe on your hard drive.  Back it up to a USB stick, move the directory around, or do whatever you like.  Have more than one dog on Dogster?  Just repeat the process again as many times as you like.  The ID number is the key.  To be on the safe side,it might be best to use a different project name each time.

Please note that I've put together this method and this writeup in a hurry.  There might be some cases where the process doesn't work.  If you run into trouble, leave me a message in the comments section and I'll try to help you out.  I'll also try to update this page based on people's feedback to clarify any spots where I haven't been clear.

Now I know that not everybody is going to be computer-savvy enough to do this.  But some will be.  All I ask is that those who can figure it out pay it forward and help out those who can't.  Back up their page, zip up the file, and email it to them.  They'll be incredibly grateful, and you'll feel good for helping them out.

Update - January 25th, 2014

First of all, I've been made aware of a couple limitations of this technique:
  • the link to the "Rosette, Star, and Special Gift History" doesn't work
  • clicking on those items from the main page doesn't give the pop-up message from the sender
  • the link to "See all my pup pals" doesn't work
  • the link to the video page doesn't work and the videos aren't downloaded
Now, I am no web guru but it looks like all these things have something in common: the links are not "static" but are dynamically generated when the link is clicked.  That is something that the web mirroring software used in this technique can't handle.  Other web mirroring software is similarly handicapped.

I was asked in one of the comments about videos and, while I can't get that to work as part of this process, I have found a workaround that will let you save each of your videos to your computer as separate files in case you no longer have the original version.

Basically, we are going to set up a "flash downloader".  This is generally an add-on to your web browser that lets you save the videos that you normally don't have an option to save.  The videos on Dogster are in something called "flash video" format, so you'll need a "player" for that too once the file has been downloaded.  I get into that as well.

Other methods besides the one I have tested below are bound to work and you are free to give this a shot with a different browser or technique, but I probably won't be able to help you much if you veer off this beaten path.  And this method should work for everybody on any operating system.

Want to get your videos back?  Here is how.

Step 1: Download Firefox If You Don't Already Have It

Why Firefox?  Because that is what I use, it worked for me, it is awesome, and it is free.  If you already have Firefox, you are awesome and can skip this step.  If you don't, download it from this link and become awesome.  Go ahead.  I'll wait.

Step 2: Open This Post in Firefox

If you are already using Firefox, skip this step as well.  If not, copy the address link for this post from your current browser to Firefox

Step 3: Get the Download Flash and Video Add-On

Like other browsers, Firefox's functionality can be extended with things called Add-Ons.
Install the Download Flash and Video Add-On from this link and clicking the "Add to Firefox" button that shows up in big letters on something like the snapshot below.

Firefox might give you a big scary warning like the one below.  No worries here.  There will be a number that counts down on the greyed out "Install" button on a window...

After a few seconds, the button will change to say "Install now" and it will no longer be greyed out.  Click on "Install Now".  The add-on will install and Firefox will tell you that it will be enabled when you restart Firefox.  Select "Restart Now".  Firefox should restart and bring you back to this page.

Step 4: Show Add-On Bar

Firefox Add-Ons are shown by default on an Add-On bar that is at the bottom of your browser window.  If you don't see a little box with a down arrow in the bottom right hand corner of your browser, hit the "Control" and "/" keys at the same time to make it appear, like so (I've enlarged it to make it easier to see).

Step 5: Get Your Video

So far, so good.  Next thing you want to do is open up your Dogster page and click on the link to open the page showing all of your videos using the "See my Video Book" link on the left side of your pet page.  What you are going to do is...
  • Start the video
  • Pause the video once it starts playing  (it seems to be important that you do this).
When you do this, you'll see the little icon in the bottom right change color.  Click on the icon and you should see something like the window below.  Ignore the "Flash Files to Download" on top.  Click on the video title under "Videos to Download" and the file should download to your computer!

I expect that the name of the downloaded file will look like some random collection of letters and numbers.  You'll certainly want to rename it to something that is understandable.

Step 6: Play the Video

I won't go into too much detail here.  Some computers might already have the software installed to play these videos.  Try opening / double clicking the file to see if it plays for you.  If it does, you are all set and can proceed with downloading all of your other videos.  If not, you'll want to install a piece of software to do so.  I recommend VideoLAN, as it is free, fantastic, and can play just about any kind of video file you throw at it.  It also runs on just about any computer out there.  Get it from this link.

          

Good luck retrieving the memories of your dog and those of your friends.  Mine are incredibly important to me.

117 comments:

  1. I'm going to have to give that a try with Chance's page.

    ReplyDelete
    Replies
    1. Good luck. Let me know if it works for you.

      Delete
  2. Thank you gonna try next week I have the whole week off...

    ReplyDelete
  3. THANKYOU !!!

    Flicka and Lucas's Mum ....Pam

    ReplyDelete
  4. Wow! This is great information! Well written and easy to understand.
    Thank you Mr. Angel Abby and Buddy's Dad!

    ReplyDelete
  5. Wow, Thanks! Gail Watson (Abbie, Charlie, Toto, Maggie, Sam, Teddy, and Prince)

    ReplyDelete
  6. Thank you so much, I can't wait to try it. I saved all the pictures and diary entries, but what I really wanted were the pages that I worked so hard on.
    Thanks again
    Deb (VT Labs)

    ReplyDelete
  7. Excellent! For those who have cats, be sure to substitute "catster" for "dogster" and "cats" for "dogs" when you change your ID number.

    Thank you very much for this write up. It is going to help a lot of the non-tech types save their stuff.

    PJ (one of the Pawsome Pages admins)

    ReplyDelete
  8. Gosh, fank yew so much fur thish - we ish Catster refugees and just got on Blogger last night - Meowmy ishn't furry smart MOL but we will pass thish on to her, yesh!

    ReplyDelete
  9. Thanks so very much this will be a fantastic help, kudos luff dem boyzzz and Mummy Sally and Angel Glen xxx

    ReplyDelete
  10. Kudos for your knowledge and help for us fur parents. You don't know just how much we appreciate your kindness to offer a way to keep our pets profile. I'd give you a kiss, but I think your wife will get mad Anyhow, thank you again.

    ReplyDelete
  11. Thank you so much for posting this information. We did not know how we could save Bailey's page and I will go back and get my other pages.

    Barbara Levine and Bailey

    ReplyDelete
  12. I did it! I can't believe I actually did it! And I'm probably one of those you mention being a few Milk Bones short.
    If I can do this, anyone can.
    THANK YOU, THANK YOU, THANK YOU!

    ReplyDelete
  13. I think you can install it on a mac, too....

    http://forum.httrack.com/readmsg/15816/15608/index.html?q=httrack+mac+version

    ReplyDelete
    Replies
    1. I mentioned that... "You'll be out of luck if you are on a Mac (sorry) unless you know how to compile your own programs." And what you'd get afterwards would be the command line version and not the version with a graphical user interface.

      If there are some Mac users out there that decide to compile it, I can pass along the instructions for how to get the command line version to work. In fact, that is how I got this working to start with. I published the approach I did because I was afraid people would be scared off working on the command line.

      Delete
    2. I'd like the instructions for a Mac. thanks!
      (I'm anonymous because I don't use any of the other accounts, but I'm a dogster mom!)

      Delete
    3. Hi Anonymous. I'm afraid you are on your own for the first part where it talks about compiling the software on your Mac in this thread: http://forum.httrack.com/readmsg/15816/15608/index.html?q=httrack+mac+version

      You'll have to get past that hurdle first according to those instructions. The next part is the magic incantation to get it to work from the command line and mirror your dogster page. I will post that in the comments section here this evening. In the meantime, take a crack at that first part.

      Delete
    4. OK. No guarantees because this is completely untested, but this should hopefully be the magic incantation if you are able to compile httrack on your Mac. From the command line, enter the following:

      httrack "http://www.dogster.com/dogs/590841" -O "590841" "-*" "+www.dogster.com/dogs/590841" "+www.dogster.com/dogs/590841/*" "+files.dogster.com/"

      This command must be all one line. Replace 590841 with the ID number for your pet. Otherwise it needs to be EXACTLY as shown, just like the Windows instructions. I am pretty sure that all those quote symbols need to be there, but not 100% sure. Note that the last four chunks of text are identical to what is supposed to be used in the Windows version. Finally, that -O is a capital "o" and not a zero.

      If you get this to work, please let me know.

      Delete
    5. Dang it. Stupid Blogger full justified the text above. Those big spaces in the command shouldn't be there. There should be just one space between each little piece of text.

      Delete
  14. Well dang, the website copier doesn't list Windows 8 - Just my luck.

    Momma going to power up the old computer to see if she can get it to work - we want our Dogster diaries. Thank you Abby and Buddy's daddy for the instructions! We love you!

    Coco Rose, Puffy and mom

    ReplyDelete
    Replies
    1. Actually, you are in luck. I think you just read it over too quickly. The download section for the website copier says "Windows 2000/XP/Vista/Seven/8". The "8" on the end is an eight :-)

      Delete
  15. You're amazing!!! It worked for me...and definitely not that hard!! Question...I will be making some diary entries before the site shuts down, so I'll need to re-save the page again. If I follow the same above procedure, will it override what I just mirrored today, or should I give it another name? Thanks again!!

    ReplyDelete
    Replies
    1. I haven't tried it myself so I can't be sure, but why chance it? I would recommend you just rename the old folder and use that as a backup. Then re-save the page again using the same name as the first time. That way you have an insurance policy, just in case.

      Delete
  16. This worked perfectly, thanks so much for sharing! The only thing that does not seem to work are the stars and gifts on the page, it shows who they are from but they no longer open to see the messages, which was a wish I had but I am certainly more than happy with the end result. Thanks again!!

    ReplyDelete
    Replies
    1. Drat. I didn't know the stars and gifts had comments so was unaware of this until now. I'll see if I can improve upon this, but I'm not that hopeful to be honest.

      Delete
    2. I spent some more time on this and I don't think I'll be able to pull it off. The way the messages on gifts work is different from the way the rest of the site works, and it doesn't look like the web mirror software (or much else for that matter) would be able to handle it. Sorry everyone.

      Delete
    3. Thanks so much for taking a second look at this! Your tutorial has been the BEST option, I have been helping others by copying their pages and did over 50 in an hour, so thanks again for sharing. The plug in for Firefox works so much better than Real Player, I am archiving at lightening speed now! You Rock!!

      Delete
  17. Thank you so much for all the work to assist the families with their furs memories.
    What a blessing your efforts to help are going to be.

    ReplyDelete
  18. It's busy downloading Sara's Dogster page. Thank you for such clear, concise instructions! You rock!

    ReplyDelete
  19. Thank you so much for this! Your directions are very easy to follow, which is great since I have so many pages to do (can't forget my angel pups and my previous foster pups!).

    ReplyDelete
  20. Fank yu SO bery much, Abby an' Buddy's Dad! My Dadda r an IT guy, tu, an' him sed,"What a great utility!" Yu toadawwy wock, mistur!

    ReplyDelete
  21. Thank you for the helpful and very detailed instructions! I downloaded Bonzer's pages and it went smoothly until the very end, where I got a flashing error message. Everything seems to be there and I can see photo comments etc but can't get the gifts to open to see the messages. Can you tell me what might have happened with my download?

    Terri/Bonzer 299237

    ReplyDelete
    Replies
    1. Oops, I just saw what you told Tracey above about the gift messages

      Delete
    2. Yes, sorry about that. I made a couple stabs at fixing that this evening and I failed spectacularly. If I knew something about web development, it might help. Keep checking back and if I get something working, I'll let folks know.

      Delete
  22. Thank you so much for sharing this with us. Now I will be able to save Sandy, Midnight, Angel Whiskey and Angel Rascal's pages :)

    ReplyDelete
  23. Crikey ... I'm going to try that when we get home from hols. Thankyou Abby and Buddy's Dad. My Secretary is hopeless but if everyone else can do it surely she can.

    ReplyDelete
  24. It worked perfectly for me, thank you so much! I'm pretty computer literate so it was easy for me. One comment... you were directing us to the blue circle you drew to show us what to do? I didn't see a blue circle but, as I said, I knew enough to figure it out.

    Ever so grateful,
    Sherri, Nick, Maya, Zelda and Angel Poana

    ReplyDelete
    Replies
    1. I'm glad it worked for you.

      The blue circle is not so much a circle, but an irregularly shaped oval that looks like it was scrawled by a two year old on a caffeine bender. It is in the screenshot that follows the text you reference, enclosing stuff that says "+*.png" "+*.gif" etc. All the stuff within that scrawled oval needs to be cleared out before going to the next step.

      Delete
  25. This is definitely much easier than the copy option that I had done earlier. Like another poster said, "You rock!"

    Although I do have some technical knowledge, this is by far one of the easiest projects I have done. Thank you!

    Hershey and Kaci http://hersheyandkaci.blogspot.com

    ReplyDelete
  26. I have a 'parallels for Windows' on my Mac, its like a virtual machine inside the mac, and I toggle back and forth between them. It is how I made the 'kimi's' that I sent to various furs. My question is, do you think this can be installed on the parallels, or do I need the full Mac version. I have zero idea about what I am doing...

    ReplyDelete
  27. Um...I need to thank you so much for this, by the way!
    And WeBeesSiamese is the same person behind the scenes as MJF aka MrJackFreckles...

    ReplyDelete
  28. How do I do this for cats? Change anything Dogs to Cats, and Dogster to Catster?
    Sorry for the dumb sounding question...LOL!

    ReplyDelete
    Replies
    1. For cats, see the comment higher up from PJ on Jan 21st: "For those who have cats, be sure to substitute "catster" for "dogster" and "cats" for "dogs" when you change your ID number."

      As for Parallels, I have never used it, nor have I used a Mac for the last 20 years. But the virtual machines allow a very faithful emulation, so there is an excellent chance that it will work just fine. Can't hurt to try.

      Delete
  29. I didn't see that comment earlier! Thanks!

    Well...we got it to work on Parallels, but then the work said there were about 330 errors, YIKES! But I hit the finish button, and got to see the whole page, but I can't get the thing to open from the desk top...its there, just can't access it.
    I may need to get the full Mac version.

    ReplyDelete
  30. Oh MY! I figured it out, I think...But it is now 3am here, BOL! Well I got home from work and after a few chores got right to it...and now all my dog/cat pages are copied! Thanks from the bottom of all of our hearts! Pawsome assistance and advice given here!
    Woofs, Purrs & Hugs!

    ReplyDelete
    Replies
    1. Good job. Please share what you did to get it working for the benefit of others.

      Delete
  31. I installed it in Parallels, just followed the directions as you have them here; and gave it a try...and it worked. Yay!
    I copied all 9 of my pages, and then had to try to figure out how to get the info when I want to 'look' at them...if you open the copymaker, you can see on the side a listing of the things you copied. And after a bunch of clicking...I figured that out too...look in 'files' click on that and voila - you get a listing of all the things copied...and when you click on any of those it will display!
    Yay!
    Thanks again!

    ReplyDelete
  32. Our Pawppy just transferred all this info into the mac side of our files...so now its even more accessible and safe, because the Mac gets backed up regularly.

    ReplyDelete
  33. Mama goofed on the previous comment. She figured out a way to see the comments under the zealies. Instead of going to the individual zealie/rosette on your home page, scroll down to where you see the link Zealie, Rosette Gift History. Click on that link and you can see all the zealies you have received and given. Click on one of the zealies and you should be able to view the comments. Problem solved.

    Love -

    Hershey and Kaci

    ReplyDelete
    Replies
    1. Unfortunately this isn't the case. Click on that link and you'll see that these pages are actually on dogster.com and not on your local hard drive. When dogster shuts down, those links will no longer work. I looked at that this morning and I think this is another one of those things that I can't fix. Those pages are generated server-side and the web mirroring software can't handle those. Sorry.

      Delete
  34. It worked fine but my video that I really wanted to save does not show up. Is there a way I can save my video from my dogster site for tiffany http://www.dogster.com/dogs/334344

    ReplyDelete
    Replies
    1. Hmmmm..... Maybe. I'll try a little later this morning.

      Delete
    2. I figured something out for this. You will have to download each of the videos separately and manually, and they won't be linked to your original Dogster page. However, you will be able to get each of the video files saved to your PC / Mac and play them anytime you want. I think that is the main thing.

      I have updated the original post that describes the process. Give it a shot and post back how it works for you.

      Delete
  35. Thank you so much! I did it - and it worked like a breeze :) I was able to save Nikita's pages - pictures, diary entries, EVERYTHING - it took less than 10 mins from "training" to "retrieving". THANK YOU! (Kathy - Nikita's Mom).

    ReplyDelete
  36. Great tutorial!

    For those having software issues, here is a one-step tool made specifically for downloading Dogster photos and videos. (no software to install)

    DOGSTER DOWNLOAD:
    http://dogsterdownload.com

    ReplyDelete
    Replies
    1. Nice!

      I gave this a try and so folks know, the diary entries show up as plain text files and the images are a bunch of separate pictures in a folder. There isn't anything like a web page that ties it all together that I could see. But for folks that can't get my method to work, this is an option worth checking out.

      Delete
  37. Thank you so very much..!!! Not only did I copy all three of my dogs pages but was able to copy the groups as well that throughout the years we played in..You are amazing and a true angel for providing us with the most wonderful tutorial..Thank you again..♥

    ReplyDelete
    Replies
    1. WOW! Can you share the set of scan rules you used to mirror the groups?

      Delete
  38. Hi there...so my first page was perfect. I repeated for dog # 2 and it seems it works but the only difference is I get this page to come up

    Index of locally available projects:

    No categories
    · Trudy's dogster page (this is a link)

    If I click the link it works and takes me to the path on my C drive.

    As for my other pet, it shows an error log. From what I can tell the log is showing the path for photos....several photos so I'm thinking I should start over?

    ReplyDelete
    Replies
    1. Did you use a different "New Project Name" for each of your dogs? You need to do that. The other possibility is that when you went to do your second dog that you made a typo somewhere, probably in the scan rules. As I mentioned, this section needs to be perfect for things to work.

      As long as you are using a different "New Project Name" for each pet, you shouldn't need to redo the first one if it looks OK. Just the second one.

      Delete
  39. Thank you so very much with all your expertise I have been able to save all three of my pups Sabrina, Abigale June and Tiffany my angel and best of all your adding how to save the videos it worked for me and now I have my precious video of Tiffany.

    ReplyDelete
  40. Thank you so much for your hard work, Abby's human! I saved all my Catster & Dogster pages last night, it was a breeze. I have a question for you, I tried to save 10 pages of a forum thread in my own group and it saved ten pages alright but when I clicked to view them it gave me that crazy Catster/Dogster message about "oops" youre trying to view a page in which you're not a member - something like that. Is there any way I can use this program to save forum threads? Thanks.

    ReplyDelete
    Replies
    1. I've tried unsuccessfully to get this to work. Somehow the mirror program needs to be told the forms-based username and password so it can pass this along to the website. Supposedly this is supported, but I tried several times unsuccessfully to get it working. Take a look at this and maybe you will have better luck than I did. Good luck.

      http://httrack.kauler.com/help/CatchURL_tutorial

      By the way, what were you using as the Scan Rules for the forum?

      Delete
  41. Thank you, I have saved our pages. This is much better then screen shots

    ReplyDelete
  42. Hi DeKay,

    This is what I was using for the Scan Rules - perhaps I did it wrong:
    -*
    +www.catster.com/forums/groups/thread/644082/1
    +www.catster.com/forums/groups/thread/644082/1/*
    +files.catster.com/
    -*
    +www.catster.com/forums/groups/thread/644082/2
    +www.catster.com/forums/groups/thread/644082/2/*
    +files.catster.com/
    -*
    +www.catster.com/forums/groups/thread/644082/3
    +www.catster.com/forums/groups/thread/644082/3/*
    +files.catster.com/

    I will read that tutorial - thank you so much for everything you're doing to help us Catster & Dogster refugees.

    ReplyDelete
    Replies
    1. To get this to work will need a combination of that tutorial and a good set of scan rules. If you are using these scan rules for a single run of the program, there are some errors here.

      The "-*" basically initializes the filters to zero. So a "-*" anywhere but at the very top of the list wipes out everything above it. You only need a "-*" at the very top. The "+files.catster.com" then becomes redundant when repeated over and over. It would just need to be there once.

      If what you have pasted here is for multiple runs, then that makes more sense.

      Good luck with the tutorial.

      Delete
  43. Thank you so much for a great tutorial on doing this, it worked great for me and all my pages are now saved. It's nice to be able to view all the photos and their comments and to see all of my diary entries too. Was fast & the best way I've seen to save their pages quickly. Thanks Angel Abby's dad. PS Give that lovable Buddy a big Hug!

    Teresa, KJ, Rain, & our Angels Kiska, Greta & Banjo

    ReplyDelete
  44. Thank you for the great instructions. I am pretty computer savvy and have volunteered to help many of my Dogster friends by pulling their pages and video. So I am paying it forward so to speak.

    ReplyDelete
  45. Just to let you know everything worked seamlessly and the video instructions were bang on. I did have to download VideoLan but it plays great. Again, thank you. Now I will go off and help all the others.

    ReplyDelete
  46. DeKay, do you know if any of the animated GIF's coding that was put into the extra bio fields on a dogs profile page will show when a person goes to view their page? Everything loads fine from My Website file (html) but on the ride hand column where I had extra content for example GIF gillter text or a GIF animated image by a (third party) those are not showing up. Do they just take a long time to load?

    ReplyDelete
    Replies
    1. I'm glad things are working for you, and thanks for helping others out. And VideoLan is indeed pretty great.

      If the GIFs would have loaded, they should have shown up very quickly since they would be stored locally. Given that you say they are hosted on a third party website, you'd have to add these websites to the end of the list of scan rules. Put these entries after the "+files.dogster.com/" entry. Each site you add will of course need a plus sign in front of it. e.g.

      +www.somegifwebsite.com/

      or somesuch. Let me know how you make out with this.

      Delete
  47. Gmail, outlook.com and office outlook will not send it as a zip file. I get an error message that "executable" files are not allowed to be sent due to security reasons. I sent it just as a regular file which of course had about 5 or 6 sub files and it seemed to go through. Has anyone actually sent the pulled pages as a zip file?

    ReplyDelete
  48. Looks like the only file you need to send if you are helping someone with pulling pages is the html. I sent it to a non-Dogster friend and he had no trouble opening it in his browser and it looked fine.

    ReplyDelete
    Replies
    1. In the example I used, you can drill down to a file that will be name 590841.html and a directory that is named 590841. Within that directory are a bunch of images and other stuff. That is the minimum that has to be sent.

      You can try this trick with zip files to cut down the size of the email: rename the extension from .zip to .piz (zip spelled backwards). Have the recipient rename it back to .zip. This usually works great.

      Delete
  49. Ok, thanks will give this a shot...the pulling pages was easy...sending them to the rightful owner has not been easy, lol.

    ReplyDelete
  50. DeKay, those above instructions worked fine, thanks for the help. Now I can help others more efficiently and get their pages to them.

    ReplyDelete
  51. Awesome!! Thank you.
    Mary Bush & The Bush Furs

    ReplyDelete
  52. DeKay, so far I have pulled 38 dog/cat pages from Dogster/Catster for those that feel overwhelmed by the task. It has worked very slick and it is way better than a static screen shot. Thank you so much for posting such great instructions and answering all my questions. I am going to miss Dogster.

    ReplyDelete
    Replies
    1. Impressive! Thanks for helping out.

      My Lovely Wife will miss Dogster as well. Seems like a lot of people are moving over to http://pets4pals.com, so hopefully that site fares better.

      Delete
  53. DeKay, this is a question that has popped up in Dogster Group Forums: I mirrored my dog's web page using HTTrack and stored on the c drive. If I clear the browser cache, disable/disconnect (not just close Firefox) from my internet, when I go the the file stored on my c drive and view it, it looks crap-o. Just text all on the left side, no photos...nothing like the mirrored page when I am connected to the internet. The question is, if I am always connected/enabled to an internet connection and Dogster is gone, will my pages look like they did when I pulled them? My pages look fine now, Dogster is still up and running and I am connected. There is a lot of confusion. If the pages I mirrored are on my c drive and I am connected to the internet and I go to the .html file and open it, everything should be good, right?

    ReplyDelete
  54. Has anyone had trouble downloading using Chrome ???
    I downloaded my friends pages and my own. My friends work great, mine are a mess, I don't know why ?????

    ReplyDelete
  55. Anonymous, sorry I am using Firefox but this is what I posted in a Dogster Forum that I found out today: Bad news for some of us. I talked to a very techie guy and he believes the reason these mirrored pages look fine is that they are pulling from Dogster itself and once Dogster is gone our pages will look like garbage. I cleared my browser cache, disabled the internet and opened the mirrored web pages for CoCo Puff and got nothing but a bunch of garbled text. Even tried the Firefox drop down menu under Web Developer and checked off Work Offline and still garbled text. Didn't matter if I opened 500107.html or the index.html...still garbled. Mirrored pages look fine when connected to the internet but I have a strong feeling that they are pulling download info from Dogster itself. Does anyone have any answers?

    ReplyDelete
    Replies
    1. Yeah, it is a problem all right. All the pages that everyone has gone to so much trouble to download are garbage and I feel terrible about it.

      However, I am expanding the scan rules to see if I can fix this. I am cautiously optimistic that I can fix this. Keep an eye on the blog for updates.

      Delete
  56. DeKay, thanks for your trouble...we are all trying to salvage something. I did get great results with Firefox Scrapbook and it definitely is not linking back to Dogster for anything. All dynamic and all offline too. I will give your new scan rules a try tomorrow and let you know how it goes.

    ReplyDelete
    Replies
    1. The blog has been updated with the new scan rules. Give them a shot.

      The nice part about my approach vs. Scrapbook is that navigation between different pages still works. But Scrapbook is a great insurance policy that people could be using just in case.

      Delete
  57. I am a cat. I tried the new scan rules. I replaced all dogster with catster and all dogs with cats. My photos don't show up. Webster said that is because the photos live on dogster *yikes* Dear Mr. Abby's Dad if you know which dogster and dogs have to stay in there and which to replace with catster and cats, would you let me know? Otherwise the new rules work PURRfect.
    Purrs and kitty kisses to you.

    ReplyDelete
    Replies
    1. If you give me your Catster ID, I'll try to look at it but I probably won't be able to get to it until the weekend if that is OK.

      Delete
  58. Dekay, firstly, you can navigate between pages with Scrapbook. For example I can go to different pages of my diary and photos. I will use this as an insurance backup, lol.
    I tried your new scan rules, cleared cache and disabled my internet connection and it looks great. I just tested it on one dog and so I figure it will be fine. Thanks for the hard work. Now I have 38 dogs to re-do, aroooooo, bol. I appreciate your help and this is a learning experience for me.

    ReplyDelete
  59. Oops -- it's Beepers again. That last line I added is not how I showed it. I started at www.

    +www.catster.com/subscr/gift_archive.php?pet_id=144863

    ReplyDelete
  60. This is Beepers from Catster. My first comment didn't post, so I'll repeat.

    Here are the Scan Rules that DeKay helped me with for saving Catster profiles. This works for Catster, to get the page and all the photo pages and comments, diary pages and comments, plus the thumbnail photos with comments:

    -*
    +www.catster.com/cats/144863
    +www.catster.com/cats/144863/*
    +tetra.saymedia.com/*
    +www.catster.com/static/*
    +www.catster.com/files/*
    +www.catster.com/photos/photo.css
    +files.catster.com*
    +files.dogster.com*
    +*.cdnsters.com/*
    +*amazonaws.com/*
    +use.typekit.com/*
    +www.catster.com/images/*

    Then I added this line and it captured the Gift History. Gifts won't open, of course, but you get a picture that also shows names of who gave the gift when you hover each gift. I don't know if this is the correct way to show this, but it worked:

    +www.catster.com/subscr/gift_archive.php?pet_id=144863

    ReplyDelete
  61. This may sound like a stupid question but when you send the copied file to someone do you also send the .whht file too? When you mirror the pages a file is also named as what you created the name of the project i.e. Hershey's Dogster pages.whht as well as the file itself.

    I have been able to copy other member's pages using this method with the updated code but we have not attempted to send any pages to people.

    Thank you.

    Hershey's Mama

    ReplyDelete
    Replies
    1. I'm about 99.99% sure that you do not need to send that file along. This file is, I believe, the file that remembers your selected options for a project. You can always rename the .whtt file to .xxx and see if the page still opens up fine and then rename it back.

      Delete
  62. This is my scan rule for See All Friends on my Lucky Kitty's Catster page. When I click on the link to see friends it goes to the next page and shows the names and containers where the thumbnail of the feline friends should be but there are no images. I must be missing something in this scan rule about getting thumbnails.
    +www.catster.com/friends_page.php?i=1116274&c=19&n=

    ReplyDelete
    Replies
    1. I can't figure out why those images don't show up either. They are hosted on files.dogster.com which is allowed in the Scan Rules, but they still don't show up for some reason. I will try to dig into this on Saturday.

      Delete
    2. I tried some more to get this to work and failed. I think it has something to do with it being a php page, even though gift history is also a php page and it works. I'm thinking to that we'd also run into problems once the list of friends went over a single page. I'll update the blog today with the other improvements made so far.

      Delete
  63. A new update went up Feb 8th that should collate all that we've learned so far and adds some new features. Check it out.

    ReplyDelete
    Replies
    1. When I tried this today, there seems to be an issue related to photobucket. On pages that have code on them (background images, css,etc.) the scan is running for hours and downloading lots of images from photobucket that have nothing to do with the dogster page. I stopped my first one after 7 plus hours, and it had downloaded more than 800 MB in 406 folders of stuff from photo bucket. The pages I did that do not include any backgrounds, etc. did not have this issue. Not sure if there is something you can do to help fix this.

      Here is an example of what it is scanning -

      Host: blog.photobucket.com
      File: /wp-content/uploads/2014/01/belly.gif
      Name: C:/My Web Sites/MacKenzie Dogster Feb 2014/blog.photobucket.com/wp-content/uploads/2014/01/belly.gif
      ContentType: image/gif
      StatusCode: 200 (OK)
      InternalStatus: 1 (receiving)
      StatusMessage: OK
      HTTP/1.1: yes
      ChunkMode: no
      TestMode: no
      HeadRequest: no
      NotModified: no
      WriteToDisk: yes
      LocalFile: no
      Size: 1271274
      TotalSize: 3919522

      And here are my scan rules -

      -*
      +www.dogster.com/dogs/210652
      +www.dogster.com/dogs/210652/*
      +www.dogster.com/subscr/gift_archive.php?pet_id=210652
      +www.dogster.com/static/*
      +www.dogster.com/files*
      +www.dogster.com/images/*
      +badge.dogster.com/*
      +files.dogster.com*
      +www.dogster.com/photos/photo.css
      +*.cdnsters.com/*
      +tetra.saymedia.com/*
      +*amazonaws.com/*
      +use.typekit.com/*
      +*mytextgraphics.com
      +*.photobucket.com/*
      +farm*.staticflickr.com/*

      Thanks!!!

      Delete
  64. Thank you sooooo much for all the info. Since being on Dogster I have lost my most beloved babies and was so afraid I would lose their beautiful pages. I now have them all tucked away furever. THANK YOU!

    ReplyDelete
  65. This is Beepers from Catster again, DeKay. Has anyone reported problems using this? I was trying to do an update on Scooter's page. The file size was 50MB a couple days ago. It only needed to capture a couple of images for her background. I saw it appear to be downloading a lot of photobucket stuff, but I let it go. She has no photobucket images but I had left in the filter for photobucket. I had to leave it for a while and when I came back, it had reached a file size of 90+MB and was still downloading. That's way too big. It was still on photobucket, I think, but I'm not sure now -- I panicked and cancelled it so I wouldn't lose the original page I had saved. I don't think I goofed this time on the photobucket filter. I had +*.photobucket.com/* I think I even used that last night without a problem. I'll run it again and take photobucket out, but I just wondered if anyone else had a problem.

    ReplyDelete
    Replies
    1. I had the same problem, I posted this above hoping there may be a solution -

      When I tried this today, there seems to be an issue related to photobucket. On pages that have code on them (background images, css,etc.) the scan is running for hours and downloading lots of images from photobucket that have nothing to do with the dogster page. I stopped my first one after 7 plus hours, and it had downloaded more than 800 MB in 406 folders of stuff from photo bucket. The pages I did that do not include any backgrounds, etc. did not have this issue. Not sure if there is something you can do to help fix this.

      Delete
    2. I have removed references to Photobucket from the post until this gets sorted out. What I would need to see is some examples of what the right stuff that it *should* be downloading from Photobucket is so I can tighten up the Scan rules, similar to the way that Flickr is narrowed up.

      Delete
  66. Sheri - abby and lucians momFebruary 9, 2014 at 10:44 AM

    Is this ok to do with all of the Bad Gateways and problems we are having posting on Dogster this weekend? Will this bring anything bad into our computer?

    ReplyDelete
  67. The directions were easy to follow but I am not having the same positive results that I've been reading about in the comments. My initial attempt at one of my dog pages took over an hour to download and it ended up scanning over 200 links. It ended up having about 40 errors. When I went to the downloaded file and clicked on the Index, the page came up just fine. Then I disconnected my internet connection and then the Index link didn't work. It said the address is not valid. When I reconnected to the internet and clicked on the Index again, it still came up with the address is not valid error. So I worked on my 2nd dog's account. Downloaded much faster although again, there were over 200 links scanned. I had the same results after downloading. Worked fine the first time with the internet connected but then didn't work after internet was disconnected or when it was reconnected. Had the same address is not valid error. I have 30+ Catster pages to work on but if I can't get the 2 dog pages to work, I'm afraid to even attempt. I'm at a loss here.

    ReplyDelete
  68. I'm having issues with an error log. I copied it. Any clue why this might be happening? When I viewed the mirrored site, it freezes up on me and one of the pictures is missing. It's really weird. Any ideas? I'm including my error log in case that helps at all...I'll need to break it up into two or three messages...

    HTTrack3.47-27+htsswf+htsjava launched on Mon, 10 Feb 2014 19:28:55 at http://www.dogster.com/dogs/776173 -* +www.dogster.com/dogs/776173 +www.dogster.com/dogs/776173/* +www.dogster.com/subscr/gift_archive.php?pet_id=776173 +www.dogster.com/static/* +www.dogster.com/files* +www.dogster.com/images/* +badge.dogster.com/* +files.dogster.com* +www.dogster.com/photos/photo.css +*.cdnsters.com/* +tetra.saymedia.com/*

    ReplyDelete
  69. +*amazonaws.com/* +use.typekit.com/* +*mytextgraphics.com* +farm*.staticflickr.com/* +farm*.static.flickr.com/*
    (winhttrack -qwC2%Ps2u1%s%uN0%I0p3DaK0H0%kf2A25000%f#f -F "Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)" -%F "" -%l "en, *" http://www.dogster.com/dogs/776173 -O1 "C:\Users\User\Desktop\dogster and catster folder\Floppy's dogster folder\Floppys dogster page" -* +www.dogster.com/dogs/776173 +www.dogster.com/dogs/776173/* +www.dogster.com/subscr/gift_archive.php?pet_id=776173 +www.dogster.com/static/* +www.dogster.com/files* +www.dogster.com/images/* +badge.dogster.com/* +files.dogster.com* +www.dogster.com/photos/photo.css +*.cdnsters.com/* +tetra.saymedia.com/* +*amazonaws.com/* +use.typekit.com/* +*mytextgraphics.com* +farm*.staticflickr.com/* +farm*.static.flickr.com/* )
    Information, Warnings and Errors reported for this mirror:
    note: the hts-log.txt file, and hts-cache folder, may contain sensitive information,
    such as username/password authentication for websites mirrored in this project
    do not share these files/folders if you want these information to remain private
    19:30:57 Error: "Receive Time Out" (-2) after 2 retries at link

    ReplyDelete
  70. www.dogster.com/robots.txt (from primary/primary)
    19:32:41 Warning: Retry after error -4 (No data (connection closed)) at link files.dogster.com/pix/dogs/73/776173/776173_1209164393.jpg (from www.dogster.com/dogs/776173)
    19:32:45 Error: "Not Found" (404) at link www.dogster.com/images/common/shimr.gif (from www.dogster.com/dogs/776173)
    19:32:45 Error: "Not Found" (404) at link www.dogster.com/images/shimr.gif (from www.dogster.com/dogs/776173)
    19:33:33 Error: "Forbidden" (403) at link a1.cdnsters.com/static/resc/jquery-ui-1.8.custom2/css/ui-lightness/images/ui-anim_basic_16x16.gif (from a1.cdnsters.com/releases/20120815-1015/resc_cdn/reset-dsterText-960-jquery.min.css.gz)
    19:33:33 Error: "Not Found" (404) at link files.dogster.com/dcmag/images/bg_trendingBar.png (from a1.cdnsters.com/releases/20120815-1015/resc_cdn/2colster-d.min.css.gz)
    19:33:33 Error: "Forbidden" (403) at link a1.cdnsters.com/releases/20120815-1015/resc_cdn/sc_cdn/images/icons/icon-16-comment.png (from a1.cdnsters.com/releases/20120815-1015/resc_cdn/2colster-d.min.css.gz)
    19:33:33 Error: "Not Found" (404) at link files.dogster.com/dcmag/images/icon-16-comment.png (from a1.cdnsters.com/releases/20120815-1015/resc_cdn/2colster-d.min.css.gz)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://www.dogster.com/dogs/776173"><img (from www.dogster.com/dogs/776173/photo/3598461)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209254420.gif" (from www.dogster.com/dogs/776173/photo/3598461)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1248566050.gif" (from www.dogster.com/dogs/776173/photo/4617377)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1248566065.gif" (from www.dogster.com/dogs/776173/photo/4617378)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1210261840.jpg" (from www.dogster.com/dogs/776173/photo/3626172)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209163771.jpg" (from www.dogster.com/dogs/776173/photo/3595977)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164375.jpg" (from www.dogster.com/dogs/776173/photo/3596014)
    19:33:40 Warning: Retry after error -4 (No data (connection closed)) at link files.dogster.com/pix/dogs/73/776173/776173_1209164393.jpg (from www.dogster.com/dogs/776173)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164393.jpg" (from www.dogster.com/dogs/776173/photo/3596017)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/77

    ReplyDelete
  71. 6173_1209164425.jpg" (from www.dogster.com/dogs/776173/photo/3596023)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164335.jpg" (from www.dogster.com/dogs/776173/photo/3596010)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164342.jpg" (from www.dogster.com/dogs/776173/photo/3596011)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/diary/Tails_of_devotion/776173_1209254420.gif (from www.dogster.com/dogs/776173/diary/Tails_of_devotion/582675)
    19:33:40 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/diary/776173_1209254420.gif (from www.dogster.com/dogs/776173/diary/Tails_of_devotion)
    19:33:41 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164403.jpg" (from www.dogster.com/dogs/776173/photo/3596019)
    19:33:41 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164416.jpg" (from www.dogster.com/dogs/776173/photo/3596021)
    19:33:41 Error: "Not Found" (404) at link
    www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164432.jpg" (from www.dogster.com/dogs/776173/photo/3596024)
    19:33:41 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164467.jpg" (from www.dogster.com/dogs/776173/photo/3596031)
    19:33:41 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164475.jpg" (from www.dogster.com/dogs/776173/photo/3596033)
    19:33:41 Error: "Not Found" (404) at link www.dogster.com/dogs/776173/photo/"http://files.dogster.com/pix/dogs/73/776173/776173_1209164480.jpg" (from www.dogster.com/dogs/776173/photo/3596035)
    19:33:41 Warning: Unexpected 412/416 error (Requested Range Not Satisfiable) for files.dogster.com/pix/dogs/73/776173/776173_1209164393.jpg, 'C:/Users/User/Desktop/dogster and catster folder/Floppy's dogster folder/Floppys dogster page/files.dogster.com/pix/dogs/73/776173/776173_1209164393.jpg' could not be found on disk
    19:33:42 Error: "No data (connection closed)" (-4) after 2 retries at link files.dogster.com/pix/dogs/73/776173/776173_1209164393.jpg (from !/pix/dogs/73/776173/776173_1209164393.jpg)
    HTTrack Website Copier/3.47-27 mirror complete in 4 minutes 47 seconds : 320 links scanned, 291 files written (9005579 bytes overall) [2615772 bytes received at 9114 bytes/sec], 8070467 bytes transferred using HTTP compression in 57 files, ratio 8%, 1.0 requests per connection
    (27 errors, 3 warnings, 0 messages)

    ReplyDelete
  72. Hmmm. The site was getting absolutely clobbered a couple days ago because the Wayback Machine is in the process of mirroring the entire site. Lots of people were having problems getting in. I'd just try it again. And you might want to add these options into the Scan Rules - I've been experimenting and it really helps the speed:

    --disable-security-limits
    --max-rate 1500000
    --assume php=text/html

    ReplyDelete
  73. THANKYOU!!!!!!!!!!!!! Crikey .... you are the BEST mad Scientist EVER!!!! One of my mates told me about your site and I'm a complete computer hopeless case and I managed to save my pages and all of my families pages (even my adult kids who are usually the ones that help ME with computer issues) with NO trouble. Your instructions were so easy to follow. THANKYOU, THANKYOU, THANKYOU!!!!!! AND what a nice bloke you are. Your wife is one VERY lucky Sheila!!!!! Such understanding ... I didn't think you blokes had it in you ... only joking!!!!!! You are the BEST!!

    ReplyDelete
  74. Oh, MY!
    Here we are again...I am trying to recopy/update my pages with the scan rules as they are here. The pages come up when I turn off the WiFi, but when I try to access a link, (like the photobook or the diary), then it says web page not available...which makes me think these won't work when the site(s) go down.
    I really hope you can help me with this...the pages are up in my blog now, but I think those too will disappear in due time:(
    Not the blog or the text but the pictures...I don't expect the links to work there so I took them out, but I don't think that will stop the things from going away later. So I want to redo my pages on the HTTrack.
    Do we need to add a scan rule to the lumina site? And I am still confused as to what to do about the Photobucket things.

    Thanks again, and I am so glad I have a blog to have my 'diary' in now...that can't be shut down unless I decide: Nope, it won't be anytime soon, LOL!

    ReplyDelete
  75. Hmmm.... I don't know. I just tried to search Dogster and I can't get a darn thing. Apparently, there have been a lot of code changes on their end, and that probably messed up my method.

    If all else fails, there is the wayback machine!
    https://web.archive.org/web/20150921074142/http://www.dogster.com/dogs/1266398/photos/1

    ReplyDelete
  76. Yes, you are right. And if we mirror the brokenness, likely the same things still won't work.

    I have saved lots of things in the wayback machine.

    ReplyDelete