June 19, 2003

dns woes

it's time for mozilla's dns implementation to be rewritten... no, i'm not talking about a real DNS resolver implementation. we just call gethostbyname (or one of the re-entrant IPv6 friendly flavors), really! but...

the current codebase is really showing its age. it's a fragile piece of multithreaded crap that suffers from layers-upon-layers of patches and an interface that is everything one should avoid when creating an xpcom interface.

with drastically different implementations for windows, mac classic, and other, maintenance is a complete headache. what we need (and what we can really easily achieve) is a single, cross-platform solution. microsoft kindly showed us the way by only supporting IPv6 via the new getaddrinfo API... who uses IPv4-mapped-IPv6 addresses anyways?!? :-/

so, the plan is to whittle the code down to a single implementation that calls getaddrinfo when supported by the host platform and gethostbyname (or whatever re-entrant flavor is available) otherwise.

the other big nasty with the DNS code is shutdown. a function like getaddrinfo or gethostbyname may block the calling thread seemingly forever, prompting the three-finger-salute. we have this problem today under linux and osx.

the solution, i think, is to spin up multiple detached (or unjoinable) background threads. each thread will call getaddrinfo and then communicate the result back to the resolver, which may hand the result off to several listeners. at shutdown, we will not wait for these threads to exit. the resources these threads access after getaddrinfo returns will be reference counted on the number of DNS threads plus one. this ensures that the background threads won't run into corrupted memory if the main thread has already shutdown. of course, on windows and some other platforms all background threads are abruptly terminated when the main thread exits. under linux, this is not the case, so we have to be careful. the only real problem with this solution is that tools like purify are likely to bitch. oh well :-/

finally, there is the issue of DNS caching. mozilla likes to cache IP addresses for the entire browser session in a hackish attempt to defeat a specific security threat. this makes many people unhappy, and generally causes lot's of unexpected and undesirable side-effects. we need to back off on this solution and come up with something smarter. what that is exactly remains to be determined... but, in the near term it probably involves a pref and a user prompt :-(

Posted by darin at June 19, 2003 12:25 AM
Comments

For a long time I've had to use IPv4 mapped addresses, because without them Mozilla tries to use the IPv6 socket for IPv4 communication, and ends up failing. (This was the case last month, I've not yet attempted disabling them again.) I supposed I should go look through the handful of bugs on bugzilla that have been marked FIXED and find the one that actually isn't...

Posted by: Eric Hodel at June 19, 2003 9:03 AM

'Do you want to risk security flaws?' is never going to go down well as a popup, or a pref...

Did anyone ever test this under RH9, or any of the other things in that bug on stuff hanging on shutdwn?

Posted by: Bradley Baetz at June 20, 2003 6:37 AM

hey bradley! why would RH9 be any different? if your nameserver is slow to respond then the application will have to wait... and if the UI thread is waiting on gethostbyname, then the UI will hang.

Posted by: Darin Fisher at June 20, 2003 10:18 AM

RH9 uses NTPL, so gethostbyname is a cancelation point, and can be interupted with pthread_ when we want to quit. In theory, anyway. See the bug for more info. that was before I had RH9, though, and I still haven't tested.

Posted by: Bradley Baetz at June 20, 2003 3:51 PM

is mozilla's DNS code the reason I keep getting "such-and-such.com could not be found" errors? I don't get the problem with other programs.

Posted by: robertd at June 21, 2003 9:33 PM

flubber dubber eat more rubber.

Posted by: b o b at June 23, 2003 3:56 PM

bbaetz: problem is we need a solution that works on Windows and other platforms. Windows does not support any notion of cancelation points. All you can do is terminate a thread :-(

Posted by: Darin Fisher at June 23, 2003 6:14 PM

Darin: Sure, but that what we want on unix too. The unix issue is that we can't terminate teh thread because its blocked, and doesn't go to see whats happening until later. I assume that thats not an issue on windows.

Posted by: Bradley Baetz at June 27, 2003 5:59 AM

Bradley: i should just post the prototype somewhere ;-) ... each DNS thread just holds a strong reference back to an object which contains the global mutex for DNS (among other things). when threads complete, they consult this object to process the result of getaddrinfo. this object may be in the shutdown state, in which case the DNS thread will exit. so if we're on an operating system that allows child-threads to continue running after the main thread has exited (e.g., linux), then the "process" will not end until each thread discovers that it should shutdown. however, the main thread will be able to shutdown immediately, and i've noticed that this is enough to make the OS think the application is finished. e.g., i get back a bash prompt once the main thread completes even though there are still child threads active. now, i haven't tested thoroughly against the new linuxthreads impl in RH9... i should verify that i'm not depending on some side-effect of the old threads impl :-/

Posted by: Darin Fisher at June 29, 2003 8:31 AM

"a function like getaddrinfo or gethostbyname may block the calling thread seemingly forever"

Do you mean "so long users get impatient" or "hangs program?"

Each DNS query by a resolver does have retry and timeout limits, although poorly configured resolvers w/ bad server configs can take a really long time.

Posted by: Ben Chuang at January 23, 2004 9:47 AM

You Must Promise. To call your mother, to help old ladies cross the road, and to turn your cell phone off at the movies.

Posted by: Dolora at January 24, 2004 10:53 AM

Whether native or not, this is obviously one of the first steps on your way to OS X. Keep in mind that often, the functionality of your code has a lot to do with how your interface is designed. How many developers have come up with great functional ideas from working with their interface or looking at their competitors'? Start working on your Aqua compliance from day one. Don't wait until the last minute.

Posted by: Helen at January 24, 2004 10:55 AM

Adhere to Layout Guidelines. Did you leave 12 pixels between your push buttons? Does the positioning of your pop-up menus make sense, and when do you use a pop-up versus a scrolling list? Are you using the right types of buttons for the proper functions?

Posted by: Constance at January 24, 2004 10:55 AM

So far in these articles, I have only dipped a toe or two into Aqua's pool. I have covered basic aspects of building an Aqua-compliant application, including the building of photo-illustrative/3D application icons. Now it's time to address other components of our Mac OS X application.

Posted by: John at January 24, 2004 10:55 AM

Not quite as entertaining as Shrek, but Dock animation can be an important and useful function in your application. For example, Dock animation is a helpful way to indicate the status of your application.

Posted by: Owen at January 24, 2004 10:55 AM

In building your amazing Aqua application, one of the most important things to consider is the Dock. There are three things your app needs to be "Dock Compliant." Now, I write this knowing that the Dock will be going through some major changes soon, but for the most part, these should still hold true.

Posted by: Christiana at January 24, 2004 10:56 AM

Adhere to Layout Guidelines. Did you leave 12 pixels between your push buttons? Does the positioning of your pop-up menus make sense, and when do you use a pop-up versus a scrolling list? Are you using the right types of buttons for the proper functions?

Posted by: Wymond at January 24, 2004 10:56 AM

Adopt Sheets. I really like the use of Sheets in OS X. The use of Sheets lets me know which window my dialogue belongs to without hijacking my system.

Posted by: Paschall at January 24, 2004 10:56 AM

To help you become a good Aqua citizen, Apple has created a few guidelines. I've put together a brief overview of them, and we'll be tackling many of them in the months to come.

Posted by: Adlard at January 24, 2004 10:56 AM

Whether native or not, this is obviously one of the first steps on your way to OS X. Keep in mind that often, the functionality of your code has a lot to do with how your interface is designed. How many developers have come up with great functional ideas from working with their interface or looking at their competitors'? Start working on your Aqua compliance from day one. Don't wait until the last minute.

Posted by: Rebecca at January 24, 2004 10:56 AM

My friend told me about your web site and I really enjoyed it. Very nicely done. Very interesting!

オンラインカジノ

Posted by: オンラインカジノ at February 14, 2004 9:14 PM

WOW!!!! This is a great website.

business money

Posted by: business money at February 24, 2004 8:44 AM