My 2009 MacBook’s speaker doesn't work (most times). The digital audio out port gets stuck. It is an old MacBook with separate headset & mic ports.
I have discounted problems with the speaker as the startup sound goes directly to the built-in speakers.
I think the OS controls which functionality the computer decides to use, but consults some hidden hardware in the socket. The System Preferences can sense but not control the state of the digital audio. As far as the computer is concerned, it is outputting digital audio and the speaker is disabled.
I did try to fix the problem by putting in a digital audio connector, but that didn’t help. I did find suggestions recommending things like poking a toothpick in the hole and trying to ‘unlatch’ some mechanism which (I assume) senses the presence of the plug.
My next step, unless I get a better idea, is to open the computer case in the hope that I can inspect the port from the inside. Alternatively, I might try to:
look further for software solutions, somehow overriding the computer's belief that something is plugged in; OS X doesn't seem to support this
mechanically probe from the outside (again)
get wireless speakers or headset
get some kind of optical portable or device
re-organise places I go (I carry a phone headset about, and in one office I keep a cheap pair of Logitech wired speakers. Some places have Apple TVs.)
something else I haven't seen yet
Any ideas about pros and cons? (I don't have much money to spend and hope to avoid the expensive options above. I have time, but want to explore more options before opening the case.)
If you look down the socket while the computer is on, and the back of the socket lit red, then I would try putting a vacuum nozzle to the socket in an attempt to pull out whatever might be stuck back there.
You are correct; the operating system normally decides what output to go with. But in this case, the chip that converts digital to analog audio on your logic board will tell OS X when it thinks a mini toslink cable has been inserted, and you will lose the option for 'Internal Speaker' in System Preferences.
If the back of the socket is not lit red, then it may in fact be software issue. Perhaps try a PRAM reset?
There is no software that is going to fix a physical issue.
Now, I have never disassembled the audio port on a MBP logic board, but it is safe to say there is a physical connection being made within the jack to tell it either an analog or an optical jack has been inserted. It may be some as simple as a mechanical contact (something like a switch) or a photo sensor. Either way, there is no logic in your headphone jack to tell your MBP "I'm optical, turn on your red light!" It only does so, because a connection was made.
Because that connection is made, your audio subsystem is telling OS X that a headset is connected and to reroute all your audio through that device. There is no software or setting that is going to trick that audio subsystem into not "believing" what it is "seeing." In this case, that connection being made. Whether what is making that connection is an actual audio device or some gunk jammed in there is another story altogether.
I wouldn't use a toothpick.
Try using compressed air to clean out the port. If that doesn't work, you can a small spritz of electronic component cleaner (I have used CRC Quick Dry Cleaner myself) Make sure you do this while the computer is off and unplugged, of course. Don't apply power until you are sure it's dry. Wait 1 hour to be sure!!! Oh...and make sure you read the directions and warnings on the label!
If that fails, have a logic board repair specialist look at it. They can probably replace the component pretty quickly and cheaply.