I'm preparing to power-off and disconnect some old Cisco 3550-48 Ethernet switches at a Customer site. For kicks, I jumped on the units and pulled the output of sho ver one last time. I was shocked, and felt like I had to erect some sort of memorial to the faithful service of these devices.

They (and the UPS devices that they're attached to) were installed in early December 2003 during an IP telephony migration. The switch in the maintenance garage (the first one) did well, certainly, but the switches in the wiring closet (the second two) performed beyond my wildest dreams. Looking back at my records, I'm not seeing that any trouble tickets were ever logged relating to these switches (bad ports, etc). They appear to be fully functional.

If you're familiar with Cisco IOS and the output of sho ver then you'll know what's noteworthy about the output below.

(I know that this is silly, but for some reason I find this very touching.)

Good night, faithful servants. You did a great job.

The Checkpoint SecureRemote / SecureClient package was the source of our woes. We were using "NGX R60 HFA1". Upgrading to "NGX R60 HFA2" caused the problem to disappear immediately.

Shame on me for not thinking about eliminating this common factor between all the crashing machines sooner. I'm dumb.

Hey, lazyweb-- do you know anything about Sierra Wireless Compass 597 USB CDMA adapters causing NT kernel "STOP" errors (aka "Blue Screen" or "BSOD") on Windows XP Professional installations?

I've got three (3) of these adapters, and four (4) different laptop computers, and I can reliably make all of the combinations of adapters and laptops stop just by plugging-in the adapter after the drivers are loaded.

The adapter has this hokey function where it appears initially as a USB CD-ROM drive (called "TRU-install"-- used to load the driver w/o requiring outboard CD media), and that functions fine. Some of the Sierra Wireless k-base articles relate to problems with this installation process, but that's not my problem.

After the drivers are loaded, though, plugging-in the adapter results in an NT kernel stop to the tune of:

DRIVER_IRQL_NOT_LESS_OR_EQUAL
STOP 0x0000001D (0x305f6469, 0x00000002, 0x00000008, 0x305f6469) 

I've managed to replicate this on a Dell Latitude D630, a Latitude D430, an Inspiron 6400, and an Inspiron 710m. All the machines have Symantec Antivirus 10.1.5.5000 installed, but I uninstalled it one one computer and observed the same behavior. Other commonalities between the machines are user-land software-- Microsoft Office 2003, Adobe Reader 8, etc. There's no other software in common between all the machines that should have kind of kernel-mode driver loaded.

All the machines were installed with Service Pack 2 and all current updates, but for kicks I installed Service Pack 3 onto two (2) of the PCs and saw no change in behavior.

(Our wireless carrier on these, BTW, is Sprint. I've downloaded the most current version of their hideous management software from their web site, and also tried to install the device without the Sprint software, just using the Sierra Wireless drivers to make it appear as a modem. No dice for either of those attempts... *sigh*)

I'm preparing to install a white-box desktop computer with a clean installation of Windows XP and Service Pack 2 (with no additional software) to see what happens. I've got another Latitude D630 to try, as well (one with a software load I can wipe with impunity).

I've been loading the drivers provided with the device in its USB mass-storage profile. Given that I can't get the PC to operate with the device plugged-in, it's difficult for me to report on version numbers of drivers. Plugging-in the device while booted in "Safe Mode", I see a driver that "cannot start" provided by Sierra Wireless dated 6/20/2008, version 2.1.7.1, signed by WHQL.

I've searched and searched, and I'm not coming up with much in the way of similar reports. Hopefully, I'll attract some other poor souls who are seeing the same behavior with this post. We'll just see...

I know that the problem isn't any of the following:

• There appears to be a conflict between the AirCard modem and the Texas Instruments PCI GemCore-based SmartCard controller.
• There is a conflict with older versions of some CD burning suites (like Roxio) and the software they install...
• Intel Centrino chipset can freeze laptops trying to run AirCard Watcher software...

I think I'm safe to go ahead and write this one, what w/ only one "working day" left in 2007. We'll see...

So, a user (we'll call her "Shelly") calls me indicating that her vertical-market time and billing accounting application is not spell checking data she enters. Further, Shelly states, she spent over an hour on the phone with the application manufacturer's technical support, and they've decided that the registry on her Windows XP Professional-based PC is "corrupt". Other users in the office don't seem to have any problem, but she can enter violently misspelled words and see no squiggly red lines warning her of misspellings.

I choked back my initial urge to comment on the use of the word "corrupt" ("What, has it been taking bribes or something?!?"), since I firmly believe that the words "corrupt" and "registry" coming out of a technical support representative's mouth together are really code for "I don't know what's wrong but I don't want this to be my problem anymore." I also choked back my urge to comment on the obscene yearly "maintenance" fee required by the software manufacturer that, apparently, entitles their Customers to receive slipshod, finger-pointing "support".

I agreed that I'd fire up a VPN connection, hop onto her PC with a remote desktop control application, and "ride along" to see what was happening.

I got connected to Shelly's PC and she demonstrated, in her characteristic ALL CAPS TYPING STYLE, the lack of spell checking. I've always cringed at the ALL CAPS-ness of her typing, but I know that her ways cannot be changed. She's been doing this since her office got time and billing accounting software back in the late 1980s.

"Did tech support have you logon to another PC where the spell checker works fine to see if the problem 'follows' your user settings," I asked. I was betting the spell-check component has both machine and user registry settings, and given that Shelly's office uses roaming profiles, a mal-adjusted user setting should 'follow' her, whereas a machine setting should stay in place.

"Uhh, no", she said. That's a strike against software manufacturer technical support, to me. I suppose they couldn't know that we were or weren't using roaming profiles, but even w/o roaming user profiles, different users on a Windows NT-derived operating system have different user registries.

I think it's reasonable to see if you can isolate an issue to being a "per-machine" or "per-user" issue at level 1. Of course, since they wrote the software to begin with, you'd think they'd already know where their settings were stored. To be honest, though, I'd guess that even their development staff don't have any idea where these supposedly "corrupt" settings are stored. My bet would be that they don't even have a way to talk to the people who wrote the spell checker, seeing as how it's a licensed third-party library.

I convinced Shelly that we should make sure the problem wasn't with her user settings. "But they said my registry is corrupt! My registry! Corrupt!" Oh, the horror. Luckily I've got a pretty good track record of solving problems for Shelly, and she let me proceed as I wanted.

I logged-on with my test user account and we attempted to reproduce the issue. My account received proper spell checking, unlike Shelly's. At least I knew that the underlying code was working fine, and likely this was a user registry issue.

I decided to head into the menus and have a look at the spell check options. Ultimately, I wanted to see where the settings are being saved in the registry. I figured I'd do my usual trick of taking a snapshot of the registry, changing a setting and closing the app, then taking a snapshot again and comparing the snapshots. (Sure, sure-- I could just run RegMon while I use the app, but I can do this faster and w/o downloading software.)

I never even made it to taking my "before" snapshot, though. When I looked at the options dialog, one (unchecked for my account) item jumped right out at me: "Ignore words in ALL CAPS".

"Umm, Shelly-- let's logon as you again and check one setting."

Sure enough, the setting "Ignore words in ALL CAPS" was checked in her settings. I read the setting's name out loud, and she replied: "Well, I set that because I type everything in all capital letters and I wanted it to..." She trailed off, then paused. "Oh," she said, "I understand. Sorry to have bothered you."

"No problem," I said, and disconnected from her PC. Nothing more could be said.

I suppose the only thing that could've made this better would've been if the spell checker had a setting to "Ignore misspelled words" and she'd enabled that.

I did a job for some friends this evening at a small dental practice replacing a server computer. I removed the "old server" and its "UPS" and had a big laugh. The alleged UPS, one of those APC "power strip with a battery" models (a Back-UPS ES 650), was providing standby power for the old server computer, or rather, was supposed to be.

When I pulled the UPS out from under the desk, I found the factory-applied bright yellow sticker advising the user to connect the battery still intact and firmly covering all the battery-backed outlets. The server computer had been plugged into the outlets labeled "Surge Protection Only" for the 5+ years that it was in production use.

I've just written a Group Policy Administrative Template for the "fix" outlined in Microsoft KB article 946627. If you're experiencing the crashes described in that KB article or the Microsoft Security Response Center blog entry, you may want to consider deploying this registry change. With this ADM, you can centrally apply this change using Group Policy.

Grab the file here: http://peeved.org/misc/946627.adm

Beware that this is not a true "policy" and that (a) you'll need to turn off "View / Filtering / Only show policy setting sthat can be fully managed", and (b) it will "tattoo" the registries of machines such that if you ever need to disable the setting you will need to create an evil "anti-policy" to disable it. Neither thing concerns me, but I thought I'd be up-front about it.

Good luck.

My business partners and I are deploying Office 2007 at a few Customer sites now, and we've quickly learned that Outlook 2007 is greatly "enhanced" with the addition of Windows Desktop Search (WDS). Without WDS, the "Find" functionality in Outlook is diminished.

With this in mind, I embarked on installing WDS as part of an Office 2007 deployment. This particular Customer was of the mind that client computers should not be indexing resources on server computers, being that they have an enterprise search product already implemented to do that. I started looking for a way to administer WDS via Group Policy and, specifically, how to disable indexing of resources via UNC or "mapped" "drive".

In version of WDS prior to 3, from what I could tell, there was a registry setting to disable indexing of any resources accessible via UNC. Further, from what I can tell by examining the EXEs and DLLs that come with WDS 3.01, the name of the registry value that controls this behavior, PreventIndexingNetworkShares, is no longer a string that occurs in the SearchIndexer.EXE executable. (As an aside, I did find the string in another file, Deskbar.DLL, but I don't expect that it would control indexing behavior.)

The Windows Desktop Search Administration Guide details extracting a group policy template, DesktopSearch30.ADM, out of the installation package for WDS, and I saw no specific setting to prevent indexing of all UNC resources. The next best setting, appeared to be PreventIndexingCertainPaths, known by the en-us localized string "Prevent Indexing Certain Paths". The syntax to specify UNC paths is odd to me, being of the form otfs://{*}/server/share/path. I wondered if I could exclude all UNC paths by specifying otfs://{*}/*/*, so I tried.

After specifying this setting in a GPO and rebooting my deployment test machine, I found that I was waiting approximately two (2) minutes at "Applying settings..." before the boot would proceed. A little searching turned up a conversation on the MSDN forums that described the issue I was having. It seemed that the Windows Search service (SearchIndexer.EXE, running as the WSearch service) was hanging up the boot process until the service control manager got tired of waiting on it. Helpfully, an Ari Polsky from Microsoft indicated that it was a "known issue", and offered two (2) mutually unacceptable "fixes": (1) To make the WSearch service dependent on the NLA service which several users indicated did not help, and (2) mark the WSearch service for manual startup, which sorta works.

I decided to go the route of the manual service start, and modified my GPO containing my WDS management policy to include a setting to make the WSearch service start manually. The next boot went very quickly, but I discovered upon logon that the service didn't start "as advertised" by Mr. Polsky's message, and that attempting to start the service gave me a "The service terminated with service-specific error 2147500053". This, too, was reported in the MSDN forum thread, and several readers figured out that marking the WSearch service for manual startup by altering only the Start parameter in the registry, rather than using the a service policy in a GPO, seemed to alleviate the service-specific error.

I knew that service policies in GPOs also alter the security descriptor assigned to the service, so using the sc sdshow command, I found that the security descriptor for a stock WSearch service was:

After modifying the service to start manually, using group policy, but not modifying the security settings, the security descriptor becomes:

These descriptors are expressed in the Security Descriptor Definition Language, or SDDL. It's painfully difficult to read, but, fortunately, some people at the University of Washington assembled a SDDL syntax guide that can help you decode these nasty strings. One thing that their guide doesn't contain, though, is a description of how the types of "rights" granted by the descriptor, which are phrased in their documentation in the manner of filesystem permissions such as "Create All Child Objects" or "List Contents", are applied to services.

Microsoft wrote a knowledge-base article, KB 914392, which describes the Best practices and guidance for writers of service discretionary access control lists. This article is dense and, unfortunately, uses abbreviations that are somewhat ambiguous to describe the meaning of the "rights" granted. Alun Jones, a Microsoft "Most Valued Professional" for security, wrote a nice article describing how SDDL rights relate to services that is understandable and clear.

So, armed with these articles, I performed an exercise to figure out what security descriptor was required in order for the WSearch service to start and run properly. Fortunately, I hit it right off the bat. The "stock" descriptor names "Built-in Administrators", "SYSTEM", "Authenticated Users", and "Power Users". The descriptor that is applied by group policy, by default, names "Built-in Administrators", "SYSTEM", and "INTERACTIVE". Just for kicks, I added "Authenticated Users" to my service policy GPO, and granted them "Read" and "Start, stop and pause" permission. This gave me a resulting security descriptor of:

That last bit-- the part that ends in ;;AU) now matches the "stock" descriptor for "Power Users" (the one ending in ;;;PU) in the first example).

One reboot later (which proceeded at normal speed), and I found that the WSearch service had started itself after the user logged-on (as indicated by time-stamps in the System event log), and did not terminate with any errors. Success! I'm not really concerned that plain ol' users can stop and start the WSearch service, because it seems to want to start itself (by way of the WDS user-land UI).

After doing all of this, I finally got a change to see if the otfs://{*}/*/* setting for the "Prevent Indexing Certain Paths" policy worked, and I'm happy to report that it does. The users will not be indexing the server computers via "mapped" "drive" or UNC.

In summary, if you're having trouble with long startup / bootup times on computers running Windows Desktop Search 3.01 with settings being applied via Group Policy, I'd recommend creating a service policy to set the Windows Search service to manual startup, and modifying the security settings in the policy to include "Authenticated Users" with "Read" and "Start, stop and pause" rights.

As an aside to this whole thing, it appears that Microsoft has implemented a Group Policy Client Extension (GUID {7933F41E-56F8-41d6-A31C-4148A711EE93}) in srchadmin.dll for WDS to process Administrative Template settings from. Searching Goooogle for keywords like CLASS MACHINE and CLIENTEXT, I'm not coming up with a lot of Administrative Template files, other than this one for WDS, that make use of the client extension capability. I've thought about doing it for some work that we've done, and after seeing how easy it is to integrate it into an Administrative Template, perhaps I should think more strongly about it.

Another in the series of "hope some other poor sod else finds this helpful" posts:

I was putting together Adobe Reader 8.1.1 for installation via Active Directory Group Policy Software Installation, and ran into a minor fun issue. After a diagnosis and resolution, I thought it'd be helpful to document it here.

I'd already obtained the full Adobe Reader 8.1 download from Adobe's site some time ago. Rather than download a full version of 8.1.1 (heck-- I don't even know if there is a full version of 8.1.1), I downloaded the 8.1.1 patch. I'd already created a transform with the Adobe Customization Wizard for Adobe Reader 8.1, and I was going to gamble that this transform would still work with 8.1.1.

I extracted the MSI and CAB from the Adobe Reader 8.1 setup program, and from there, performed an administrative install of the MSI using MSIEXEC /a AcroRead.msi. I stuck the administrative install point onto a server computer and used MSIEXEC /p ReaderUpd811_all_incr.msp /a \\path\to\MSI\AcroRead.msi /qb SHORTFILENAMES=TRUE to apply the patch to the administrative install point.

I setup an installation in an existing GPO to target the upgrade to my test deployment machine only and fired up the test machine. To my displeasure, however, a quick inspection of the event log after logon on the test machine showed the following:

An examination of the program files\Adobe\Reader 8.0 directory confirmed that there was no Update subdirectory. An examination of the program Files\Adobe directory, however, revealed a READER~1 directory. Ah ha!

I moved my faulty administrative install point aside, re-extracted an administrative install point from AcroRead.MSI, and re-applied the patch without using the SHORTFILENAMES=TRUE argument to MSIEXEC. I bounced my test machine and watched it properly install Adobe Reader 8.1.1 over Adobe Reader 8.0. Success!

My "conventional wisdom" in applying patches has been to blindly provide the SHORTFILENAMES=1 argument to patch installations. Obviously, this isn't always the correct thing to do.

Jeff and I were working tonight on some code that needed to search an Active Directory domain at its root, using the python-ldap library. We had a problem wherein our search would work successfully if performed at a base DN below the root of the domain, but not at the top of the domain. The error returned in the python-ldap exception was: LdapErr: DSID-0C090627, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, vece

Normally, this error indicates that you're attempting to bind anonymously, which Active Directory (sensibly) doesn't allow by default. We were supplying credentials to bind, though, and changing the base DN on the search to a sub-OU was all that was necessary to get the search to work. It turns out that python-ldap was binding anonymously, so the error was only sort of a red herring.

Using a sniffer, we determined that the python-ldap library was chasing referrals being returned by Active Directory to our Configuration, ForestDNSZones, and DomainDNSZones NCs. I think that setting a LDAP_SERVER_DOMAIN_SCOPE_OID server control would have probably stopped Active Directory from returning the referrals in the first place, but it appears that python-ldap doesn't support server controls "yet". As such, we opted for adding the following to our code (right after importing 'ldap') to stop python-ldap from chasing the referrals:

We still get the referrals in our results, but we can wade thru those ourselves.

Thanks to Derek Anderson for a quasi-related posting that finally got us in gear...

This post is another in my ongoing series of postings about things that might help other "IT people".

I just built a "BartPE" bootable Windows CD to recover a Microsoft BKF-format backup image for a PC with a failed hard disk drive. I was surprised to get NTBACKUP running under the "BartPE" environment fairly easily.

My CD was built using the Windows XPE 1.0.7 plugin on top of a Windows Server 2003 SP1 slipstreamed CD. I copied NTBACKUP.EXE and NTMSAPI.DLL from a Windows XP SP2 machine into my "custom" folder and burned that onto the CD.

NTBACKUP comes up with an error writing its log file, and complains about the Removable Storage Service being inaccessible, but other than that it works fine. I'm quite pleased.

Off to restore a failed PC now...