Tuesday, January 15, 2008

The registry

Just finished reading the comments in my 'Big changes' post; there are some interesting comments about the registry in there.

Could the registry be replaced? Sure, in software, pretty much any change is possible.
In practice though...If you were to replace the registry, you would end up with something that pretty much looks exactly like the registry.

Go take a look at all the programs on your hard disk that use the registry APIs in [consults wikipedia] in advapi32.dll. (There is a way to dump the APIs that a binary uses with link.exe; wish I could remember the command right now.)

Pretty much every executable file uses the registry, and expects keys to be in certain places in the registry. The APIs for getting/setting stuff in the registry are not going to go away for the next 10 years.

What about making a brand new son-of-registry, with new APIs, that new apps will use?
This will fix a bunch of problems (because you would be able to sidestep the mistakes you made while designing the current registry; add more metadata to keys to not roam, etc)

...but it would be decades before application developers will use it. (Because nobody is going to target their apps to Windows 7 only; they would wait before Windows 7 was the oldest OS out there before relying on that API. A lot of our external application developers only use APIs that have been in since Windows 2000. Sigh.)

21 comments:

Unknown said...

The solution to this problem, in my mind, is that Microsoft needs to build a stronger exclusivity incentive into the next version of Windows for developers. For example, most applications for the Mac are written to run on the current release and the previous release of the OS, which means Leopard and Tiger. Most applications I can think of for Windows are made to be compatible with Windows 2000, XP, and Vista, if not Win9x stuff as well. That's more than 10 years of compatibility you're making yourself accountable for. If Windows 7 doesn't really have any major architectural changes, why not build something really incredible into that product and use that as an exclusivity incentive not only for consumers, but for developers too? Give everybody some concrete to want Windows 7. Make it desirable. I do sincerely understand why compatibility is crucial to vitality of Windows, but you can't run with the wind of innovation when you're tied to the sails of tradition. If Vista breaks so much stuff, why not build Windows 7 to only have backwards compat with just Vista? That would take care of a lot of cruft, would it not?

Unknown said...

Eric: Your last thought doesn't make too much sense...if Windows 7 is compatible with Vista, 7 is in turn also very compatible with XP/2K because Vista itself is very compatible with those. Hence "running with the wind of innovation" would require a huge break in compatibility with ANY Windows that is out there.

Steve said...

aaron: I think that he simply means to be compatible with Vista specific software (ie software that doesn't run on Vista with the backwards compatibility 'layer').

Anonymous said...

I personally think the only way to 'deal' with the legacy of the Windows Registry is to finish what Vista started with file and Registry access virtualization of protected region and virtualize the whole darn thing. I don't see why existing apps can't continue to use existing registry APIs that are hooked (much in the same way that ShimEng.dll and ApLayer.dll hook CreateProcess etc on Vista) so that the actual replacement backing store for the registry couldn't have proper transactional semantics.

Indeed taking this further (and speculating wildly about where MSR projects like Singularity etc might go) why not do what Apple did to support OS 9 on OS X and virtualize the whole Win32 API. You wouldn't need the binary translation and instruction set emulation parts - but it would let MS, finally, move beyond Win32. Presumably by Windows 8 or 9 we'd be in a position where 64 bit was common place - so you'd have the address space needed to really allow such a compatability layer to 'breathe'....

Unknown said...

@aaron axvig, steve is exactly right, but let me refine that point a bit more. The great thing about Windows is that it has maintained compatibility with so much legacy software in spite of its advances over the years. Flip the coin over, though, and that same asset becomes more and more of a burden with each successive Windows release. With all the layer-model engineering that went into Vista (see Larry Osterman's post about layer court) , the Windows team has got to have a better grasp than ever of how to elegantly remove old stuff that needs to go, and deprecate old stuff that should never be used again. Furthermore, knowing that should give them much clearer ideas about transition pathways for developers to use less crufty API. Microsoft needs to draw a line in the sand with Vista and make Windows 7 a platform that developers will want to write against and that consumers will want to buy new, upgraded "Windows 7-compatible" software for. Of course, it may take till Windows 8 to make that transition happen, but it needs to happen at some point, and the sooner the better in my mind.

tuxplorer said...

Maybe MS can change the way the registry is loaded into memory or something along similar lines so that at least performance is not hampered? I currently install most new apps I haven't tried including free and retail ones in a virtual machine and I also backup the registry using ERUNT (which is an excellent tool) and then restore it when I uninstall an app installed directly on the system.

Or maybe MS could use registry monitoring and virtualization to aid complete cleanup when that app is uninstalled? Apart from the other overhead, the large registry size in Vista is also what hampers its performance.

As for keeping Windows 7 attractive, they should make it attactive from the developer and CONSUMER side. Develop some nice high-end pro type apps and keep them locked and downloadable for Windows 7. No bundling issues, no crying "Where are the killer apps", no argument about the incentive to upgrade.

bluvg said...

I commented in the original post, and the "son-of-registry" idea is pretty much what I was suggesting.

Someone else mentioned virtualizing the registry, and that is also what I was thinking--existing apps don't know they're dealing with anything new, but the back-end architecture could change significantly. What about something along the lines of the IIS metabase? But I would suggest making it distributed with predictable paths, rather than lumping it into either a system or user file. Put the app-specific registry file under the app's folder in Program Files, and the user-specific app registry files in corresponding folders under the Roaming folder in their profile. When you go to uninstall something, you can be sure that it is fully uninstalled by simply deleting folder in the Program Files, plus you can simply drag-and-drop folders for installation, with the settings already configured. MacOS-like, yes, but without sacrificing the benefits of the registry, which would exist as a subsystem that manages the lot. Plus, put in some PowerShell intelligence, and you solve some tricky management issues--for example, you could edit offline user profiles (roaming or local) without using login scripts or other methods. Or quick and dirty, you could edit it with Notepad.

I also like the idea of a "default settings" regeneration option or automatic backup copies (perhaps via VSS), where you can delete the registry file and it gets regenerated with either default settings or reverts to a backup automatically. And if something gets truly hosed, you just copy a "known good" registry file for the app from another computer, network share, etc.

You would need to market this appropriately, though--you need to make sure that people know that "it's not the 'registry' anymore" yet it's backwards compatible. Call it something else--it has a psychological effect. If people think it's "a warmed-over registry," they'll be much less enthused. If it's a totally redesigned approach for settings storage in Windows that sets a new bar for all OSes--yet it contains a backwards compatibility layer that truly works--then you've got a huge, huge winner on your hands.

Snirky said...

I don't really understand the intricacies, but the registry has always been a PITA. I used to love my Amiga. Copy the program from one drive to another and it works... no installation needed, just put it there, double click and it works. I believe any version of Windows is overly complex. I haven't used my Mac in a while, but it's easier there, too. Want to move Photoshop to a different disk? No problem, drag, drop, and it runs.

Steve said...

@technoid: there are licensing issues with the 'works wherever' approach. I'm not sure exactly how Apple solves this (if they do at all), but I think that if licensing security was implemented in an elegant way on Windows, it would be a nice feature.

Viking said...

Soma, so the registry replacement would have to look like the registry because no one wants to change their apps.

Oh and the 64bit model is LLP because nobody wants to change their apps.

And the windows driver model will stay a marginally prettier wrapper (UMDF/KMDF) over a dead horse (WDM) because nobody wants to change the way they do their drivers.

And bits of the desktop UI (quicklaunch) are somehow identified as part of IE (docs settings\user\local\app data\ms\ie\quicklaunch) and that'll stay that way cause "yes mr. commissioner, IE is part of windows".

You know what, i actually don't care much for microsoft anyway, so by all means stick with that approach and all the best... :)

Viking said...

I agree kinda, and nobody claims it would be easy to fix past mistakes or simply move onto a better way of doing things.
But if you want to make an omelette you gotta break some eggs.

Plus, if you make 80% profit on each copy of windows; then maybe just maybe don't burn it all on consumer electronics and pretending to have colored balls, but instead find a way to move forward. Alternatively, feel free to stay mediocre and reduce the price, but you can't have it both ways.

Old apps could get a virtual per-user-per-app registry to talk to while the rest of windows moves on...whatever - there's a thousand approaches.

As for the whole "devs won't use your features cause they'll stick with the classic stuff that works everywhere" shtick. Sorry, but that is just the worst possible attitude. Plus i think you're example of DX10 is a good one, because it shows the patience that's required. If dx10 versions of games are better than dx9 versions of games and if devs find it easier to code to dx10 than dx9, then there's nothing to fear. It's just a matter of patience.

Where do YOU think windows will end up the way it's going with it's
.) 64 bit binaries in "System32" and 32bit binaries in "SysWOW64"
.) a registry with each hive having a sub-folder-clone of itself for the other bitness
.) a 64 bit environment where longs are 32 bit
.) a single file is used to keep a user's every preference for every application installed on every computer (roaming)
.) *insert your favourite pet legacy hate here*

In the end devs want to make the least changes just for the sake of a new version of Windows - true. But devs also want to program a clean and well designed, logical environment.
So you're substituting short-term shrill screams (thing-x has changed!), for constant grumblings (god i hate windows...)

sorry bout the length.

Viking said...

Thanks for the reprieve on the verbal diarrhoea .. :)

I agree again with what you're saying. There's no direct short-term money incentive for suits to make these kind of decisions.... Just like politicians don't have a direct incentive to put in a policy or change that will have a short term negative effect, but a huge positive effect more than 4 years down the track...

On the other hand this is a blog supposedly by a windows dev with some anonymous "no-holds barred" opinions. An engineer - passion for excellence and all that gumpf. But his viewpoint seem to reflect the management thoughts quite a bit which is... disappointing. Then again i suppose you can't expect management to have one agenda and it to not filter down to the code monkeys.

As for Microsoft / Windows being liked or disliked by x percentage of people - there's no way to determine this accurately either way so i'll just drop that point...

In summary i agree, it's the tempting thing to do (or rather not do), but i strongly disagree that it's the "sensible thing to do money-wise". Particularly at a point in time when you're positively rolling in money and talent.

Dave said...

I've read a lot of stuff (mostly all the same in different tastes) about this WinMin, and I really hope you guys at Microsoft are really doing it!
For the registry thing, why don’t you do like Apple does? I mean , if this microkernel is really that fast and good as they say around, just virtualize a XP or Vista like ambient (with all the stuff, registry included). I mean, apparently Microsoft is working hard on virtualization, with Virtual PC and other things. It would be easy to “force” (ask?) AMD and Intel to put special virtualization instructions on their processors. Once the developers and the market sees how much performance Windows 7 has, they will come to you praying to develop drivers and programs that runs natively on the new kernel.
Isn’t time to start with a new, solid and fast OS? Geez you have so much resources to do this… just do it! Time to put the 1990s programs in a closed sandbox that runs inside a whole new system. And screw the compatibility with a stupid printer made by some unknown manufacturer.
But I guess it’s a matter of having the guts or not having it too
By the way, keep working! I really do like Vista and I’ll keep supporting the Win family!

Dave

Viking said...

Apple's OS X does not have a microkernel, it has a monolithic kernel. To my knowledge there are no OSes around based on microkernels.
The reason for this is that microkernels in their current design, have very large performance penalties.

btw, anyone who replies with GNU HURD should be voted off the internet.

Dave said...

Yes, I know that OSX has not a microkernel. My English’s not as good as I want and so I failed to explain me.
What I wanted to say about Apple is that when they moved to a new system from os9 to osx, they just included the classic mode to support the old programs (ok, not in leopard anymore). And also when they switched from Power PC to Intel they just build Rosetta... If MS hypothetically decide to start fresh with a new base, why couldn’t they do like Apple did?

Unknown said...

Go back to .ini files.

At least with an .ini file you could move an application on the hard disk without breaking it.

Viking said...

Sorry Dave i miss-understood what you meant.

I do think you might see Apple's move from 9->10 as simpler than it was.

1.) - Classic environment. This is where they stuck a copy of OS9 into OSX and get it to fire up whenever you run an OS9 app.

Problems: Security; As the system started up a copy of OS9 for the user, everything running in OS9 had root access to the filesystem. OS9 didn't have a full security model or permissions so this was generally simpler to do.
Can you imagine having to look after two OS' file permissions on disk. Two sets of user accounts, etc... Or alternatively everything on the PC runs under the same account in the windows-classic environment... also not nice.

2.) Carbon. You seem to have forgotten Carbon. This is where they retooled and trimmed the old OS9 Toolbox API's to make then work in OSX.

Problems: They wanted to just provide the classic environment and get developers to code for the native OSX API called Cocoa. The developers told them to get bent. So instead they had to modify the Toolbox environment to fit OSX. This took them literally years to do (i think around 4 years), particularly the integration with Cocoa.

Lastly, Apple's scenario back then was a lot different to windows' predicament now. I agree they need to finally start moving towards the future and leave old crud behind, but i don't know how much of Apple's OS migration is useful to Microsoft's current situation.

Paul said...

Why not do something innovative to keep the backwards compatibility and at the same time be able to move Windows 7 forward with many modernizations, such as updating/modernizing the Registry. What comes to mind is the Rosetta compatibility built into OS X Tiger ad Leopard (or even the 68K emulator (which has been said to have saved the company) built into the MacOS when Apple moved from the Motorola 6800 family of processors to the PwerPC processor. This allowed Apple to break from its past to modernize its hardware and software while still retaining compatibility with older software.

Microsoft seems to have difficulty modernizing its operating system, and it shows (i.e. the Registry). If Microsoft would say, as Apple has more than once, that we will only take backwards compatibility so far, then there might be a chance the next version of Windows might be as good, or better, than OS X, Unix and Linux. Modernize the Windows OS. Don't be so concerned if the process of modernization breaks compatibility. Do as Apple had and build into Windows a compatibility layer similar to Rosetta compatibility layer which allows programs written for PowerPC Macs run on Intel Macs. I believe 90% of all PowerPC software runs fine (if a bit slower) on Intel Macs.

What is holding Windows back as a modern OS is its unwillingness to give up backward compatibility with older software. There doesn't have to be a compromise between modernizing the Windows and retaining backwards compatibility. I am running MacOS 10.4 and have a few programs which are 10+ years old which run perfectly.

Your blog makes many references to Apple, the Mac and OS X. Apple must be doing something right for you, a Windows OS developer, to devote so much space in your blog discussing Apple, the Mac and comparing Windows to OS X>

Unknown said...

first off as far as this new registry only being set for windows 7 and not other windows versions what microsoft would need to do is have a registry emulator (sort of like a windows emulator)only designed specifically for the registry. In addition to both the new registry (repository without .exe files and the virtual registry emulator which supports .exe files designed for previous windows generations) what needs to be done is to have both of them password encoded. That way if a virus does want access to the registry it will have to know the password to gain access to either the repository or the virtual registry emulator (This way it solves the reverse compatability issue without falling back on previous window design flaws and old source codes to do this)And this new registry can avoid using all .exe files making way for newer and better technology to come without being held back by legacy issues. The virtual registry emulator with password encoding is what is required to do all this. And password encoding for the new repository as well. Virtual emulators work for multi operating systems so why cant the same concept be given here to this registry repository approach

Dave said...

Paul, that was just what I was saying!

Anthony Lawrence said...

I think Apple is much smarter by using XML files. They can be compiled to a binary format for speed if really necessary (though nowadays, how often is it really?) but you can always convert them to text and the point of text is that you can always fix it with a text editor when you have to.

But Microsoft screws that concept up always. Outlook PST files, as another example. Mail should be stored as individual text files or as one big text file with separators (though I dislike that) and separate index files. Then when the index breaks (and indexes ALWAYS break), they can be rebuilt without danger of losing anything.