I mean, background that reminds me of Motif defaults and NCSA Mosaic, fonts from Hell, and UI elements that look like they can fall off the screen and cause some considerable damage to my feet. With the help of the /etc/X11/app-defaults/Emacs and ~/.xemacs/custom.el its ugliness can be reduced to this:
The custom.el file has the line about KOI8 Cyrillic environment commented out -- uncomment it if you need it. Some fonts are Monotype fonts from "corefonts"/"msttcorefonts" package -- install them or change font names to something you have.
Please note that XEmacs uses X fonts while GTK and Qt use Xft fonts -- it means, XEmacs can't use font smoothing, and you have to make sure, font is included in X server configuration (shows up in xfontsel).