Disallowed types of applications revealed by Apple include
Software that changes the native user interface elements or behaviors of Mac OS X.
Software that does not comply with the Apple Macintosh Human Interface Guidelines.
Software that is similar in look or function to current Apple products (e.g. Mac App Store, Finder, iTunes, iChat, etc.).
Software similar to other software that is already released in the Mac App Store. Examples given: Adobe Illustrator and CorelDraw, Photoshop Lightroom & Apple Aperture, Cinema 4D and 3D Max, etc.
Software that contains or displays pornographic material.
Software that is or installs shared components (kernel extensions, browser plugins, QuickTime components, etc.).
Software that provides content or services that expire.
Software that does not run on the currently shipping version of Mac OS.
Beta, demo, trial, or test versions of software.
Software that references trademarks unless the developer has explicit permission to use them
Open source software licensed only under the GPL (because the App Store Terms of Service imposes additional restrictions incompatible with the GPL)
Apps that use software libraries that are either optionally installed or deemed deprecated by Apple for Mac OS X users. Examples given:
Apple's implementation of Java SE 6 (although the OpenJDK implementation of Java SE 7 is permitted if bundled into the app)
PowerPC code requiring Rosetta