Spotlight hung on OS X Yosemite with 100% CPU usage

I recently upgraded from OS X Mavericks to Yosemite, and all went well except for one thing: The mds processes were constantly taking 100% CPU, resulting in decreased system performance and decreased battery life. Now I’ve been through this dance before, knowing full well that every OS X upgrade results in a Spotlight re-index to upgrade the Spotlight DB and pick up any new files. But in the past this re-index had taken at most an hour or two on my roughly 80% full 750GB SSD drive. But this time was different — Spotlight was running not for hours, but for days. I decided to let Spotlight run thinking that maybe things were different with Yosemite. After all, Spotlight was one of the big new changes highlighted by Apple in their Yosemite reveal. But after 4 days of 24×7 indexing, I decided that enough was enough and started to investigate.

There are a million different things that can go wrong with Spotlight, and plenty of articles out there on how to fix them. Some of the things I tried were:

  • Verify Disk & Verify Disk Permissions in Disk Utility
  • Fiddle with the mdutil command to disable/re-enable indexing
  • Manually removing the /.Spotlight-V100 directory so that it would be re-created
  • Adding / to Spotlight’s Privacy tab and then removing it to force a re-index.

I tried them all, and none worked.

The symptoms were simple: the mds process was constantly taking 100% CPU. I had a sneaking suspicion that it was hitting a file that it didn’t know what to do with and would just get stuck and not proceed. I was right. Using lsof I checked what files the mds process had open:

sudo lsof -c '/mds$/'

Amongst all the various /System/Library, /.Spotlight-V100, and /Volumes files and directories, I noticed a file that stuck out like a sore thumb:

/Users/MyName/Documents/Subfolder/SomeApp.app/Contents/MacOS/Flash Player

It was actually listed twice in the lsof output. Weird, lets go check it out in Finder. Sure enough, trying to navigate there in Finder caused Finder itself to freeze up at 100% CPU usage. So I restarted Finder with Option-RightClick-Relaunch from the Dock and manually removed the file from Terminal. I then re-started Spotlight and told it to re-index from scratch:

sudo rm -rf /Users/MyName/Documents/Subfolder/SomeApp.app
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
sudo mdutil -i on -E /

This time I could see progress in the Spotlight window’s progress bar, and see files in the Spotlight DB on disk being changed and the total size increasing. About 45 minutes later Spotlight had indexed all 580GB of data. FIXED!

I have no idea why both Spotlight and Finder has trouble with this file. It was never an issue on previous versions of OS X. It was an old custom app built by a friend of mine in 2010. And while mds was stuck on the Flash Player within it, Finder crashed when navigating to directory containing the app itself. Weird.

Some helpful commands for when dealing with Spotlight:

Stop Spotlight:
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

Start Spotlight:
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

Get the size of the Spotlight Database:
sudo du -m /.Spotlight-V100

Get a list of files opened by the MDS process:
lsof -c '/mds$/'

UPDATE

Turns out this troublesome file made its way onto my Time Machine backup as well, causing the *exact same issue* in Time Machine. Backups would start and get stuck on this file, causing the backupd process take up 100% CPU forever until I manually killed the backup. So if your Time Machine is exhibiting similar behaviors as described above, find out what file(s) it is barfing on with:

sudo lsof -c backupd
or
sudo lsof -p [PID]

Then use Time Machine’s “Delete All Backups Of…” functionality to remove it. Kick off another run of Time Machine and it should make it all the way through.

If an Apple Engineer comes across this blogpost and would like a copy of the troublesome file, hit me up.

How to increase bash shell history length in OS X

I do a lot of command line work on my OS X machine, so the history saves me a lot of time running repeat commands and also refreshing my memory on commands that I haven’t run in a while. Unfortunately, the default history length in OS X is 500 commands. That seems like a lot, but when you’re running 50+ commands a day it can push older commands off the list pretty quickly. This is easily solved by setting the HISTFILESIZE in your .bash_profile file.

First, to find out what HISTFILESIZE is currently set at, run the following command from Terminal:

echo $HISTFILESIZE

To change this value, simply add the following line to your .bash_profile file (found in /Users/yourname/):

HISTFILESIZE=2000

This increases your bash history to 2000 items. It will take effect next time you open a Terminal window.

OS X Snow Leopard Bugs: Audio gets reset to mute on reboot.

While my upgrade to OS x 10.6 Snow Leopard has been mostly smooth, there are a few annoying bugs I’ve come across. This one has to do with audio – upon reboot, audio gets set to mute no matter what settings I had the volume on before. Not a big problem, but annoying none the less. Here’s how to fix it:

  1. In Finder, navigate to Macintosh HD > Library > Preferences > Audio
  2. Delete the following files:
    •      com.apple.audio.DeviceSettings.plist
    •      com.apple.audio.SystemSettings.plist
  3. Open System Preferences > Sound
  4. Set your sound to your preferred settings
  5. Exit System Preferences and Reboot

There you go! After deleting those files, they will be re-created when you go into System Preferences and your audio will no longer be muted upon reboot.

iPhone Wishlist – Form Input

Added to my list of iPhone annoyances that I hope are someday fixed:

When entering text into a form input, the first letter defaults to uppercase. This is fine for textboxes and most other inputs, but it is annoying for username/password inputs because more often than not these fields are case-sensitive. Perhaps the iPhone could be more intelligent with the auto-caps by disabling it when you’re on a text input named “username” or “login.”

iPhone / Gmail / Google Calendar wishlist.

I’ve had my iPhone for about a year now, and while I love it and the fact that it has made me more productive, it is still not perfect. There are still a few things which give me a headache, so I thought I’d compile a wishlist here for no reason other than my own sanity.

To start off, here is the context under which I have compiled this list:

  • Running Windows XP
  • Using Gmail’s hosted mail for my domain
  • Using Google Calendar for all of my events and time tracking
  • Using iPhone 3G, firmware v2.2.1
  • Using iTunes v8.0.2 to manage and sync my data

My main goal is to have all contacts & calendars sync seamlessly between all of my devices: my desktop workstation, my iPhone, and my online Gmail/Google Calendar accounts. To break it down, starting with contacts:

Contacts
When I first got my iPhone, I synched my contacts with Outlook. This is annoying to me because I hate Outlook. It’s also an unneccesary step because I don’t use Outlook for my email, I use Gmail. So while I had a decent backup of my iPhone contacts on my desktop machine, this didn’t help at all with my daily Gmail usage. In order to keep things in sync I would have to manually export from Outlook to CSV, then import into Gmail. Who wants to manually export/import contacts all the time? I do so much emailing that my contacts change at least weekly, if not daily. Undoubtedly I’d give up on a regular manual sync, causing my contacts to became a clusterfuck of chaos.

Then to my surprise, I noticed that iTunes has an option to sync with Google Contacts. “Yay, my prayers have been answered! I can finally ditch the middleman (Outlook) and my life is complete!” Or so I thought. The problem is that it syncs all of your contacts, even Gmail’s “Suggested Contacts,” which I never use. The sync worked flawlessly, with the unfortunate side effect of filling my iPhone with hundreds of random email addresses, most of which I don’t know who they belong to and I will ever need to contact again. So here is my wishlist to make my iPhone/Gmail experience perfect.

  • iTunes: Allow me to chose which contacts to sync from Gmail. A checkbox of groups would be ideal. If I could disable the syncing of the “Suggested Contacts” group, things would be perfect. Or…
  • Gmail: Allow me to disable the “Suggested Contacts” feature. I don’t like it, don’t use it, and it messes up my iPhone sync. Just add a simple checkbox to settings that allows me to enable/disable Suggested Contacts. How hard is this to add?

Calendars
First off let me say that I love Google Calendar. I use it to track everything from friend/family birthdays to my work hours as a freelance contractor. The problem here again is that I am reliant on Outlook as the middleman to sync all of my calendars between my Google Calendar and iPhone. I have installed the Google Calendar Sync app, and it works great between Google and Outlook. iTunes then syncs with Outlook so that I have events synced to my iPhone. The problem here is that Google Calendar Sync only syncs your main calendar. I have all birthdays in their own “Birthday” calendar in Google in order to keep them separate from personal/work events. So when I sync calendars, my iPhone doesn’t contain any birthdays because Google Calendar Sync only syncs my main calendar. It sure would be nice to have birthdays on my iPhone! I could move all of the birthdays back into my main calendar, but then they’d be intertwined with my work hour tracking and personal events, resulting in a big confusing mess. So here is what I need in order for this process to be perfect:

  • iTunes: Direct sync with Google Calendar so that I don’t need Outlook and Google Calendar Sync. Also, let me select which calendars to sync. Or…
  • Google Calendar Sync: Allow me to select which calendars to sync.

Overall
Overall, I wish that iTunes would sync directly with Gmail and Google Calendar, with the option to select which Contact Groups and Calendars to sync. That way I could ditch the middlemen (Outlook and Google Calendar Sync). I’m sure that these changes will come eventually, but I am throwing a digital temper tantrum here because I want these NOW NOW NOW! What is so frustrating is that these seem like trivial changes that could be made with minimal work. Google has provided APIs for Contacts and Calendars, so why doesn’t iTunes use them to their full potential? Probably because they want me to pay for MobileME. Also, why doesn’t Gmail give users control over their annoying and useless Suggested Contacts feature? Does Google know better than me? Reminds me a lot of that old dinosaur Microsoft. It drives me crazy that fixes which are so simple and obvious haven’t been implemented a full year and a half after the original iPhone was released.

If anyone has any suggestions to tackle the items I’ve listed above, I’m all ears. I know there are better syncing apps out there, but I don’t feel like paying $40 to accomplish something that I feel should be standard in today’s mobile world. Switch to Mac and use iCal? Sure, I’ll get around to that someday. MobileMe? I’ve thought about it, but would rather use a simple and free solution.

iPhone AIM App: How to switch accounts.

iPhone AIM AppThe newly released AIM client for the iPhone is pretty cool, but it still needs some improvements to be a killer app. One issue I’ve come across is how to switch between various accounts. There doesn’t seem to be a way to store multiple AIM logins and switch between them quickly. The only way this seems possible to is edit your login infomation. The hardest part was figuring out where to edit the settings. This is because your AIM settings are stored inthe iPhone’s Settings page rather than within the AIM app. So here is how to switch accounts using the iPhone AIM App:

  1. Click the “Home” button
  2. Tap the “Settings” icon
  3. Scroll down to the “AIM” section and tap it
  4. Edit the “Screenname” and “Password” to match the account you want to log in
  5. Tap “Settings” at the top to return to the Settings screen
  6. Click the “Home” button to return to your Home Screen
  7. Tap the “AIM” account to launch the app

That’s the only way I’ve found to switch accounts in the iPhone AIM App. Hopefully AIM will update their app to facilitate this process, and hopefully they will keep it within the AIM App. It seems silly to me that you have to quit the AIM App to edit your settings. It would be better if all AIM settings could be edited directly within the app.