will not boot - stumped

[alexlauf]

i can not boot from my hard disk.
1. it registers ok ( from what i can tell ) in bios ok.
2. it gets to verifing dmi pool, and stops.
3. i tried another hard disk ( both it and problem win 2000 ) it boots ok
4. as slave drive, with alternate hard disk as master, , i can access the drive.
5. with a disk editor, partition sector looks ok.
6. made modification to partition boot to jump directly to missing operating system. still hangs , no
message displayed.
it seems like if it ever tried loading the partition sector, i should have seen that message, so unless i
am missing something about the boot process, it seems it never gets around to reading the disk. but
if so, why not ? seems like if the bios was mis-set, it would not be able to be read when mounted as
the slave drive. what am i missing here ?

drive is western digital WD800JB. OS ( if it matters ) is win 2000 sp4. MB is nforce3-a.

[Geek-9pm]

A hard drive can be perfectly normal but still not bookable. The recovery console in Windows XP has two files to fix his.
FIXMBR
FIXBOOT
You have Windows 2000, which has fix-up programs with the same name. But they are different.
To get to the recovery console, you have to boot the Windows install CD and find then option to use the recovery console.

(Vista and Windows 7 are different.)

[alexlauf]

thanks, i already knew about those, but there did not seem to be much point, as it never loaded
either the MBR or the boot sector. i got desperate, and tried it, and fixboot fixed it !
the boot sector still looks the same, so i am wondering what got fixed ? does windows have protected
or read-only sectors, or am i thinking about some old system ?
i am happy to get back my disk, but this is going to bother me until i figure out what happened !

[Geek-9pm]

This forum helps with practical problems. As to how and why Windows does things, it is hard to say Witherspoon source code. Looking at the machine code is of little value unless you can get a very broad viewing what happens at boot time.

Years ago a I made a very simple boot loader for a Floppy controller  I made. Others in the company could not understand how it worked because I neglected to document it. When they asked me, I was able to provide and answer immediate, because I was the designer of the hardware and firmware. and software loader. And that system was much, much simpler that Windows or DOS.

The point is, without an official explanation of how it works, everybody just makes a guess as to how it works.

What tools don you use to examine the boot sequence? To really see what happens you have to have special equipment tat logs every instruction execution. This is a hardware debugger the runs in real-time. The average user doesn't have one of them thar whatchamacallit.

[alexlauf]

i did it the way we did things before there were such things as hardware debuggers ; i made a small
modification in the boot loader so it would jump and display the "missing operating system" message,
if it ever actually ran it. since it never displayed it ( or anything else for that matter ), i could tell it never
made it to loading and executing the code.

the way i modified the partition sector was with a home brew program using createfile and normal
read and writes. that turned out to be the cause ; it reads writes the data ok, but it seems to do
something ( to the sector ? ), so it doesn't seem load at boot time. and the bios seems to be too
dumb to output a message to indicate what is happening. but it reads and writes fine, so it took
a while to realize that I had somehow caused it. that is the problem with doing this stuff on your
own ; you occasionally get hoist on your own petard !

i remember something about protected or read-only sectors, but i don't remember if that was even this
platform or an old trs-80 thing. can't find anything on the net so far about it. it seems though, like there is
something about the sector that got changed other than the data itself, so it could be read and written by
my simple efforts. but that messed it up at boot.

what is "Witherspoon source code" by the way ?

[patio]

A Reece Witherspoon movie trailer perhaps ? ?

[patio]

What HomeBrew method did you use to modify things ? ?

[alexlauf]

a c++ mfc program. basically, i opened the device using createfile, read in the sector, made a change, then wrote
it back, then closed the device. the program also has a hex display & code for moving around and making modifications to the disk. it is a work in progress, although, i have to do some research now why it doesn't
seem to write the partition sector quite right.

this is all done on Win 2000. i don't think this will work anymore for later OS's, but it's what i have ( and
can afford ). for that matter, it is debatable if it works completely now, since i'm pretty sure it is what
made the boot partition not load.

i was stalled out because i want scrolling, as 32 bit scrollbars and 64 bit sized hard disks don't mix. i was
possibly going to write my own scrolling, but it looks like i probably have to go back and rework the basic
disk i/o instead. or learn to live with a disk editor that can't do the partition sector.

[Geek-9pm]

Witherspoon source code

My metaphor has caused confusion.  Witherspoon stars in the move 'Source Code.'

Sorry, did not want this to become a movie review. Great movie. If you like far out. Has a cleaver ending.

What I meant was that it is hard to know what a windows program really does unless you are really deep into it with extreme tools.  Some, eave all, BIOS calls can be intercepted by the OS. To workaround this, software engineers have things that hackers only dream about.

Don't be surprised that things do not work as you might expect down at the very low machine level code that could potentially alter the OS.

However, you now have the OS working normally?

[alexlauf]

yes, after i ran fixboot, it came right back.

[Geek-9pm]

yes, after i ran fixboot, it came right back.

Do you mean fixbnoot worked and later you did a reboot and windows 2000 did not come up? Did you get a specific error message or just a black screen?

Have you used the WD HDD diagnostic program?
http://support.wdc.com/product/download.asp?groupid=606

[BC_Programmer]

i was stalled out because i want scrolling, as 32 bit scrollbars and 64 bit sized hard disks don't mix.

instead of setting the scrollbar min/max and value properties directly to the hard disk size, use a scaled value, so that it fits within the scroll bar range.

Also, regarding your writing to the boot sector, remember that while reads from a physical device work in a buffered manner, usually you'll need to use FILE_FLAG_NO_BUFFERING in the call to CreateFile() in order to be able to properly write the sector data to disk. Also, when you are doing direct disk I/O, you have to seek, read, and write in multiples of the sector size of the device, and on sector boundaries. you'll need to be using DeviceIoControl() with IOCTL_DISK_GET_DRIVE_GEOMETRY to get the bytes per sector, sectors per track, number of sectors, etc. so you can compute the size of the  buffer you need to use.

[alexlauf]

i thought it was clear from the various postings, but, here it is the full sequence.

1. was working with my home brew program to edit hard disks. i used it on my boot sector. it seemed ok.
2. later when i re-booted ( quite a while, this thing runs for days sometimes ), it would not come up.
got to the verifying dmi pool message, and it stopped there. no extra beeps or messages.
3. tried to figure out problem, including things like swapping disks, changing bios settings. one thing i tried
was i modified the boot to display a message if it loaded, not realizing the disk editor was probably the
cause of the problem ( i had forgotten using it before the crash ).
4. no message resulted from above so i concluded it was not loading the boot sector. at this point i was
stumped. did not see any point in trying to fix the boot sector, since the data itself was ok. i then tried
this forum for help.
5. was told to try recovery console fixmbr and fixboot.
6. my initial thought was phooey! since the data was ok, what was the point of trying to fix it ? but i got
desperate enough to try it.
7. i ran fixboot. then i rebooted.
8. that fixed the machine. it booted after that.
9. my conclusion is i had done it to myself, and my program does something to the sector / disk that
i have not currently figured out.


about the program itself;
1. "instead of setting the scrollbar min/max ... use a scaled value" - i thought of that, but i do want it to
be 1-1, 1 byte to 1 click. otherwise, just mapping 1 scroll unit to 1 sector, @ 512 / sector, i could get to
either 1 or 2 TB sizes ( i forget if scrollbars are signed or unsigned right now ). that would probably be
enough, if that was what i wanted.
2. as for seek, multiples of sector size, boundrys, , devicei/o, etc, i did all that, with the exception of the "FILE_FLAG_NO_BUFFERING" flag. i assumed it flushed everything on closing. other than the central
mystery involving the boot sector, it seems to do ok. might not be a bad idea to put it in, if i keep using
this method. but unless i figure out what is going on with writing the boot sector, i might have to find
another way to do it.

[Geek-9pm]

WOW! You are a brave soul! 
You have already verified and tested hour code on other drives? Specifically, have you tried much smaller, much oder drives with more 'conventional' drive geometry?
Part of the problem is the definition of hard drive sector location is still haunted by the legacy definitions that go back to IBM's old ideas about how you find a lotion on the hard drive.  I think I used to know what that was, but strains my brain trying to remember how it goes.

EDIT: The link below is just none of many, many references to the headache of sector translation. The problem never went away.
 Likely you are aware of it.
http://www.dewassoc.com/kbase/hard_drives/resolving_drive_barriers.htm
The 512 byte sector no longer exists. Not on large hard drives.
http://en.wikipedia.org/wiki/Disk_sector

[alexlauf]

it has been tried on 3 drives so far, an 80 GB ( the one i crashed ), a 500 GB, and a 340 GB that i am trying
to diagnose a file system problem with. i have no plans for anyone else to use it, so i don't need to be
really rigorous with the testing. basically it works, until it fails, then i fix it.

if i stay with sector boundrys and sector sizes ( the one reported to me by DeviceIoControl ) it seems to
work ok, other than that #$#%@^ boot sector. i suspect that it would work on any drive the OS can handle.

i just seek to ( sector size ) * ( assumed sector number ), and it finds it. none of that stuff with no sector 0,
which track is it, or any of that. using createfile and treating it like just another file, a lot of that sector location junk gets handled by the OS. it really doesn't matter what real sector i am in, as long as the same call to seek can put it on the same spot consistantly. the drive might translate sectors around, but if it is transparent to
the OS, i don't really care what it is doing either. after all, the whole point of an OS is to make it easier to
work with the thing, not to send our money to microsoft. let windows do all the dirty work. i'll worry about
cases where windows can't deal with it, when it happens.

earlier on, i wanted absolute control over which sector, track, head i was at, but the point of this thing is
drive exploration at the level the OS sees, so i went with the easiest course. later, if i get into any deep
drive work, where it matters exactly what track, sector, head i am at, i might tackle that again.

i have heard 512 no longer exists, but if the OS is happy thinking it is still 512, i can work with it. i
go with whatever it thinks it is using.

i'm not so much brave, as old. i used to write low level format assembly programs for my trash-80. and
in those days low level was LOW LEVEL. if i could overcome my AADD long enough to learn write a kernal
mode program, i'd be working with the drive controler for this stuff, instead of having windows do the work. .

of course, if i could do that, i'd get myself up to date, and get a good computer job somewhere too.

by the way, how does this forum assign levels ? i could have sworn i picked something higher than noobie ?
my knowledge may be scattered and piecemeal, but i've been around the track a few times. or was that
reflecting experience with this forum ?

[BC_Programmer]

no message resulted from above so i concluded it was not loading the boot sector. at this point i was
stumped. did not see any point in trying to fix the boot sector, since the data itself was ok. i then tried
this forum for help.

I found this documentation alongside WriteFile/WriteFileEx, in the MSDN:

If you write directly to a volume that has a mounted file system, you must first obtain exclusive access to the volume. Otherwise, you risk causing data corruption or system instability, because your application's writes may conflict with other changes coming from the file system and leave the contents of the volume in an inconsistent state.

With that in mind, use 0 (exclusive access) for the dwShareMode argument to CreateFile.

Another consideration for exclusive access might be to explicitly lock the volume using DeviceIoControl() and FSCTL_LOCK_VOLUME.

(the MSDN states that Windows XP is the minimum supported client, but that isn't true, it's supported back to NT 3.5)

using createfile and treating it like just another file, a lot of that sector location junk gets handled by the OS.

No, it doesn't. Directly accessing a volume means you don't want the OS to do things for you. Writes/reads have to be aligned on sector boundaries and the buffer written or read to/from needs to be a multiple of the sector size.

it really doesn't matter what real sector i am in

Well.. if you want to write to the boot sector, I would imagine that is rather important.(although I take it you mean 'real' sectors as opposed to those translated by the disk, which would of course not really matter, as you note) Also, direct physical access goes past the actual Filesystem driver, which may also be relevant.

after all, the whole point of an OS is to make it easier to work with the thing, not to send our money to microsoft. let windows do all the dirty work. i'll worry about cases where windows can't deal with it, when it happens.

The way you are opening the volume bypasses both the file system driver as well as any interfacing between windows and that file system driver; you are dealing directly with the functionality that the file system driver would use.

i'm not so much brave, as old. i used to write low level format assembly programs for my trash-80. and
in those days low level was LOW LEVEL. if i could overcome my AADD long enough to learn write a kernal
mode program, i'd be working with the drive controler for this stuff, instead of having windows do the work. .

This is something of a problem. You are trying to deal with a co-operative, pre-emptive, protected mode Operating System the same way you did with a real-mode Operating System. The latter exposed everything to you, and you could change anything. The former does not, by virtue of being a multi-tasking environment, and applications need to share resources. Additionally, there are 'checks and balances' to prevent copius changes to the system, thus the hoops for exclusive access and sector alignment (because you are bypassing much of the functionality of the OS).

Another important consideration: NTFS keeps a backup of the boot sector. I'm not sure how it is employed, but it might be used in some manner and that somehow prevents the boot. That is assuming you are using NTFS.

Also, the symptoms you describe are usually caused by a corrupt MBR. Since the first instance was almost certainly caused by your application, I don't see any reason to think that the second write (to add the JMP to output the message) was more successful.

Consider comparing what your application sees with what DSKPROBE sees. DSKPROBE is a tool that can be found on the Windows 2000 CD-ROM.

[Geek-9pm]

Thanks, BC.
Wish I could have said that. Yes, the Windows OS is now very mean and limits some tings you do.
For reference, this gives some tips on using the MBR.
http://thestarman.pcministry.com/asm/mbr/BootToolsRefs.htm
One issue not well  documented is about the sectors that come after MBR and before the first partition. Some systems use that area for some kind of extension to the MBR boot process. Some references deny that. Or ignore it.

[alexlauf]

reply for BC_Programmer :

With that in mind, use 0 (exclusive access)...

I probably should have done this, but my first inclination was to try to
co-exist with any other things wanting access.

explicitly lock the volume using DeviceIoControl()

i did not think of this. although when i have the boot sector open,
how much real traffic should there be to that sector ? it is no
real surprise that it only showed up when i rebooted. the same can
not be said about other sectors though, so i guess i should do the
locking bit. also this was originally to be mainly for the dvd rom,
so locking was not really an issue, since i did not plan any writing.

lets not drag the dvd into this, by the way ; it has its own quirks,
some i am still figuring out, and if we start on that, this will become
the thread that never dies.

Writes/reads have to be aligned on sector boundaries ...

buffer a multiple of the sector size.

i did these, at least as far as i could tell. i used DeviceIoControl
rather than GetDiskFreeSpace to get the disk and sector size. i wonder
if they might be disagreeing with each other ?

checking the literature, my buffer address probably is not alligned in
memory to a sector boundry. seems like if this was part of the problem,
it would either give back bad data, or not work, like it does when not
sector alligned. although i might try and fix this later and see if it
crashes again, now that i know what to do about it when it won't boot.   

"it really doesn't matter what real sector i am in"

if you want to write to the boot sector, I would imagine that is rather important.

what i meant was, for instance if i go to the start of the disk, and it shows me
the boot sector consistantly, i don't care if some drive logic is really showing
me some sector in the middle where it remapped the sector to because the real
start sector had an error. if the OS thinks it is at sector 0, and presents me
with the boot sector, thats good enough for me. if the thing was not handling all
the junk like sectors starting at 1, chs or lba, and similar, it would not be
working at all. and microsoft would be telling me to use something diffrent.

Also, direct physical access goes past the actual Filesystem driver, which may
also be relevant

some stuff may be bypassed with this method, but there must still be a good hunk
of the filesystem between me and the drive. part of the design philosophy was
to be able to treat EVERYTHING like a file, at least with this part if the API.
personally i'd rather have drive specific API calls.

You are trying to deal with a co-operative, pre-emptive, protected mode Operating System

I know, that is one reason i did not go with exclusive access.

NTFS keeps a backup of the boot sector.

I know, although i have yet to locate it. it's position varies depending on the
specific OS. I don't think this is a factor here ; it seems to have failed
during the bios sector loading. if bios was aware of an alternate boot sector
copy, it should have used it, and I'd never have known my main sector was bad.
I suspect that bios us unaware of the copy and it is there for use by the OS,
probably for fixboot, and similar. and it possibly might have come into play
if it had booted, and the OS saw the odball boot sector ; but it never made
it that far. 

Also, the symptoms you describe are usually caused by a corrupt MBR.

i agree, but as i said, i kept looking at it and it kept looking ok. both the
before and after reads of the data looked identical.

Since the first instance ... caused by your application, I don't see any reason
to think that the second write ... was more successful.

It was just as successful - both failed. and that is what told me boot up was
failing before running the boot sector code. as to why, since the OS ( when
running ) and my app see the data just like it should be, i did not suspect
my stupid app was the cause. why the data can read and look ok with the OS,
but not by the bios loader, is still unanswered. and you would think the bios
would display a message or beep or something when it hit a problem.

Consider comparing what your application sees with what DSKPROBE

i would love to. part of the problem is DSKPROBE won't show me any physical
drives to select. I don't know if it is a quirk in DSKPROBE, or more likely,
of my system. for that matter, if DSKPROBE had worked, i would not be writing
this thing in the first place.

or maybe i might still have done it just to do it. i won't be really happy,
until i can do ANYTHING with this system, although typically, everyone else
will probably be running windows 47 ( or Win 2047, depending on the marketing
guys ) on organic computers by that time. 

---------------------------------------------------------------------

reply for Geek-9pm :

http://thestarman.pcministry.com/asm/mbr/BootToolsRefs.htm

neat link. I'm going to try some of the stuff from it. a lot of it looks like
the sort of things i like to play with.

 

[BC_Programmer]

some stuff may be bypassed with this method, but there must still be a good hunk
of the filesystem between me and the drive.

Nope. NTFS.SYS, FASTFAT.SYS, etc, are completely out of the loop. In fact they use the same functions (well, the native version of those functions at least) to perform their own operations.

part of the problem is DSKPROBE won't show me any physical
drives to select.

weird. You could try another alternative program, Although I've not used it for a while (I've not needed a direct disk editor beyond the curious for a while) you could try Hxd. It really is quite feature-filled.

Basically, what I meant is that since the program can't seem to write it properly (the sector is obviously changed, since it won't boot) than there isn't much reason to have faith that it is reading properly, either. (thus the idea to use another software tool).

As to the issue regarding locking, it's not so much that there could be a contention issue with other applications writing to that location as well, but sharing issues could easily cause these sorts of issues, particularly with something as sensitive as direct disk editing. Best to make sure the entire operation is atomic and completed before allowing anything else access to the drive.

[alexlauf]

sorry about the delay, life keeps interfering with my computer time.

Nope. NTFS.SYS, FASTFAT.SYS, etc, are completely out of the loop.

let me try to clarify what i meant. i misspoke a bit when i refered to the code as
"filesystem". the idea is to be able to treat  EVERYTHING like a file. by that i meant
the basic file functionality is there, and some basic ops ( open close, read, write,
sometimes seek ) can be used. that is why one call "createfile", is being used for
so many things in the first place. you are probably correct about NTFS, FASTFAT being
out of the loop with physical drives, but they are also not needed in this context.
there is no need for keeping track of free / used clusters, handling streams,
etc that filesystems normally handle. you should be able to open, read, write,
etc, without needing to care what it is you are working with. for example, you could
open a tape drive for output, open a hard drive for input, and dump the whole thing
to tape ( tape ? what's that ? ) with the same program, changing only the actual
device names. they do this all the time in unix / linux. i am actually surprised that we
still have to worry about something as hardware specific as sector boundries & sizes. i
havn't tried it, for example, but i would practically bet that if i read past the end of the
drive i would get an eof error .   

program can't seem to write it properly...
isn't much reason to have faith that it is reading properly

it is obviously doing something wrong ( maybe, see below ), but as i said, i did the
write. later when i did a read, the data was the same. it was writing it correctly somewhere
or somehow, and it seemed  to be able to find it again later ; otherwise, i'd have gotten
garbage back. maybe that somewhere is the wrong place ( due to the buffer alignment problem )
and it crashed depending on exactly where that write ended up.

news :

1. i tried to duplicate the crash, with another drive. of course, i can modify the boot sector
all i want, and it won't crash for me when i want it too ! sometimes i hate these things. if
it really was my app that did it ( still the prime suspect ), there is some other variable
somewhere. since i did not fix my buffer to a sector boundry ( or to anything ) , i wonder
if its moving, being at a good address some days, and a bad address on others ( like the
day when it crashed ).

2. i was all ready with sectorinspector dumps with before and after modification, but i see no
difference, and as said, it didn't crash.

3. i downloaded Hxd. it looks neat. i'll be trying it out next time i try to get that thing to crash.

[Geek-9pm]

Legacy standards are still an issue with Hard Drives that are used in Personal Computers. In tis context, a computer that may conform to the standards set out by IBM. If you want to use any other kind of computer, it is another matter.

The OP needs to wade through this kind of stuff to grasp the problem. It is still with us. Some utilities still go by the partition and cylinder foundry thing.
http://www.cgsecurity.org/wiki/Menu_Geometry
There are more like these. But we could get really buried in a ton of old documentation that still belabors the PC Hard Drive definitions. Modern drive geometry is not longer sector, track, cylinder. It is whatever the manufacture wants it to be for the most effective, fast, reliable way of writing and reading data.
EDIT:
On to put it another way, sector 37 may be right after sector 36 physically.  Or maybe not. It is wherever the firmware assigned it. Tracks not longer have uniform sector count. Maybe that track does have 37 sectors. But the net track might have 35.
One of many references.

However, current disk drives use zone bit recording, where the number of sectors per track depends on the track number. Even though the disk drive will report some CHS values as sectors per track (SPT) and heads per cylinder (HPC), they have little to do with the disk drive's true geometry.
http://en.wikipedia.org/wiki/Logical_block_addressing

[BC_Programmer]

sorry about the delay, life keeps interfering with my computer time.

Same here, actually! Losing one's job tends to complicate life, that's for sure!

that is why one call "createfile", is being used for
so many things in the first place.

There is a fundamental difference here; everything isn't being treated as a file, but rather as a stream of bytes. That may seem like splitting hairs, but I feel it's an important distinction. a File is a component of storage used by a file system. A stream of bytes is just that- a stream of bytes. Windows Sockets, Named Pipes, and so forth are all streams of but not files. Presumably that is really what you mean when you say file, but I've always associated a file as a named entity of storage within a file-system (or papers in a folder ), so I feel that calling it a "file" is a bit ambiguous.

there is no need for keeping track of free / used clusters, handling streams,
etc that filesystems normally handle.

Well of course not! you are "below" that stuff at the level of direct access to the volume. You don't have to deal with them not because the Filesystem is handling it for you but because you don't have to; if you wanted to interpret and read individual files, then you would have to read and interpret that data yourself if you wanted to read the file(s) by using the direct access method.

Legacy standards are still an issue with Hard Drives that are used in Personal Computers.

No they aren't. Unless you have a legacy machine. The legacy "standards" aren't an issue with the hard  drives, they are an issue with the system BIOS in older machines. Cylinder/Heads/Sector addressing has been obsolete for nearly 20 years. Any tool still using those is in fact about that age. LBA is how hard disks are addressed now, but this is even lower than what they are dealing with. Also, as he points out, Sector remapping at the drive level is at the drive level and therefore would be invisible to the OS and thus his application; also, I don't think Sector 0 is remappable even via firmware for a number of reasons, but I'm not sure about that.

[Geek-9pm]

    Legacy standards are still an issue with Hard Drives that are used in Personal Computers.

No they aren't. Unless you have a legacy machine. The legacy "standards" aren't an issue with the hard  drives, they are an issue with the system BIOS in older machines. Cylinder/Heads/Sector addressing has been obsolete for nearly 20 years. Any tool still using those is in fact about that age. LBA is how hard disks are addressed now, but this is even lower than what they are dealing with. Also, as he points out, Sector remapping at the drive level is at the drive level and therefore would be invisible to the OS and thus his application; also, I don't think Sector 0 is remappable even via firmware for a number of reasons, but I'm not sure about that.

Wrong. It was not resolved until 1996. Many are still using PCs and Hard drives made before that year. Fixing  the issue 20 years ago was a pipe dream.
Strop smoking that stuff, BC!

[BC_Programmer]

Wrong. It was not resolved until 1996. Many are still using PCs and Hard drives made before that year.

First, you state "many are still using PCs and hard drives made before that year".... what is your source for this? Where are you gathering this information?

The VAST majority of PCs happen to be running a modern copy of Windows or Linux, so your use of "many" is clearly an attempt at weasel words.

And if you had read my post more thoroughly you might have noticed that bit where I said "Unless you have a legacy machine."

He isn't using a legacy machine, or a legacy hard drive, so this is all a redundant sidebar of redundancy.

[patio]

And if you had read my post more thoroughly you might have noticed that bit where I said "Unless you have a legacy machine."

He isn't using a legacy machine, or a legacy hard drive, so this is all a redundant sidebar of redundancy.

Exactly...