Questions about dd

[Stiganator]

The xbox I have has a 250GB HD I want to clone it a smaller drive so I can use the HD in my computer. The 250GB drive is only using 175 GB. Can I dd it to a 200GB drive? I'm using a FATX capable linux distro, and it has been going for about 3 hours, just wondering if it's hanging or if it just takes a long time.

Athlon XP 3200+
2GB RAM
Both PATA drives are Master.

 

[drag]

No, that won't work.

DD doesn't care about file systems or files or anything like that. It's not that smart. It's just copying the harddrive block by block. It doesn't care if there is any information there or not.. 0's cost as much to copy as 1's. it's going to produce a faithfull duplicate of that harddrive.

With your file system your going to have files spread all throughout the harddrive. Some will be in the beginning, some will be in the middle, some will be in the end, and others are going to be spread around the entire drive... so with 50 gigs lobbed off the end of the drive image your going to lose a lot of stuff.


What you want to do is shrink the partition of the drive small enough that you can just dd that partition to another drive. That should work 'ok'.

But what would probably work better is to format the second drive and then mount both of them. Then use the 'rsync' command to copy the data from one drive to another. Rsync uses a special algorythm that ensures that the copies of the files are not corrupted. That way your actually only copying the data and not the entire drive.

So say you set them up so that they are mounted /mnt1 and /mnt2. With /mnt1 being the original drive. Then you can go:
rsync -av /mnt1/ /mnt2


Rsync is a bit odd that it uses that trailing backspace....
rsync -av /mnt/ == copy the contents of this directory
rsync -av /mnt == copy this directory


Rsync is also special because it only copies the differences of files. Say you have to shut your computer off half-way through the copying and it's not finished. Then the next day you can start off again and it'll go through and checks the files and fix any differences and finish copying them. So you can use it to keep a mirror of data on multiple machines... it'll use ssh encryption when copying stuff over a network.

 

[drag]

Oh, rsync is only realy usefull if you just want to copy the data.

for it to be used as realy clone for Microsoft OS-stuff disk it'll probably be better to shrink the partition and then use dd.

 

[Nothinman]

I'm using a FATX capable linux distro, and it has been going for about 3 hours, just wondering if it's hanging or if it just takes a long time.

It's not hung but eventually it'll die because it ran out of space on the smaller destination drive.

 

[Brazen]

Originally posted by: Nothinman

I'm using a FATX capable linux distro, and it has been going for about 3 hours, just wondering if it's hanging or if it just takes a long time.

It's not hung but eventually it'll die because it ran out of space on the smaller destination drive.

yeah, dd takes FOOOOORRREEEEEVVVVVVEEEEERRRRR

 

[Nothinman]

Well it's as fast as the drives let it run but if you're copying 200G of data that's going to take a while no matter what app you use. =)

 

[Brazen]

Yeah, I don't know, maybe it's just my impression, but it seems like doing "dd if=/dev/zero of=/dev/hdX" to a drive takes a lot longer than copying an equivalent amount of data. I guess I don't see how it could though; like I said, it's probably just my impression.

 

[drag]

The speed at which you copy is a bit of a illusion. There are always tricks to make it seem faster then it realy is.

But besides that dd probably uses small buffers or something like that that may not be the most efficient on modern hardware. Remember it was created to convert different data types when doing backups to different sorts of tape drives.

By default dd copies in 512byte hunks. Bujt that's pretty small when copying very big files (like harddrives)

I tried it on my laptop drive, bumping the bs up to 4k and 16k on a 2gig partition to /dev/null. It didn't increase performance any (my laptop was maxed out at 45MB/s or so), but it did cut down my cpu usage from 35% to 14% (on a speedstep slowed 800mhz)

 

[acaeti]

YEah upping blocksize on DD really helps a lot. Cuts literally hours out of drive copying if you have a drive of any size. I have usually used:

#dd bs=32678 if=/dev/hda of=/dev/hdb

or similar

 

[Robor]

Not quite on topic but is it possible to use 'dd' to dump or restore an image?

 

[Nothinman]

An image? If you mean a filesystem image, sure but since dd doesn't understand the filesystem it'll copy everything including unused blocks so the image will be a lot larger than if you had used tar, partimage, etc.

 

[Robor]

Originally posted by: Nothinman
An image? If you mean a filesystem image, sure but since dd doesn't understand the filesystem it'll copy everything including unused blocks so the image will be a lot larger than if you had used tar, partimage, etc.

Yeah, we were trying to use a Ghost CD to dump an image of a Dell desktop system. The problem is Ghost was very flaky 'seeing' the SATA DVD we booted from so it would boot the system but not launch Ghost. We got around that launching Ghost from a USB thumb drive but then Ghost wouldn't see our USB external drive where we were going to dump the image.

Would 'tar' or 'partimage' be something we could use as a way to dump and image and restore that image in the future?

 

[Nothinman]

Would 'tar' or 'partimage' be something we could use as a way to dump and image and restore that image in the future?

If it's a unix system tar would work but you'd have to setup and mount the partitions before extracting the tarball onto the target. You'd also probably have to run 'grub-install(hd0)' to fix the MBR as well. Same with partimage AFAIK. Clonezilla has potential to be a totally free Ghost replacement but I don't think it's there just yet.

 

[Robor]

Originally posted by: Nothinman

Would 'tar' or 'partimage' be something we could use as a way to dump and image and restore that image in the future?

If it's a unix system tar would work but you'd have to setup and mount the partitions before extracting the tarball onto the target. You'd also probably have to run 'grub-install(hd0)' to fix the MBR as well. Same with partimage AFAIK. Clonezilla has potential to be a totally free Ghost replacement but I don't think it's there just yet.

Ah, these are WinXP Pro boxes I'm talking about so I guess that's not an option for us. I did manage to get an image of the box but I had to use Acronis which is not our 'corporate standard'.

As always - thanks for the help!

 

[acaeti]

DD would've copied the entire disk. If you had, for instance, booted from a linux bootable cd (say Ubuntu), you could

#dd if=/dev/hda of=/path/to/network/share/image_name

or

#dd if=/dev/hda of=/path/to/usb/disk

and then re-dd that out to another disk. It'll grab the mbr and everything.

 

[Nothinman]

Ah, these are WinXP Pro boxes I'm talking about so I guess that's not an option for us. I did manage to get an image of the box but I had to use Acronis which is not our 'corporate standard'.

Yea, tar wouldn't work well there. Although partimage probably would be fine since the MBR is likely the default if all they're running is Windows.