I finally found some time to sit down and get this thing working. I'll write up a more detailed Howto eventually, but I wanted to get this out there for those that want it working now. Also, if anyone figures out how to create a route based version of this, please PM me. I could not make the route based configuration work.

There are a couple of requirements:
1. You must use certificates. The Mac VPN client will send its internal IP as the IKE ID if you use PSK's. This doesn't work well for clients that get a dynamic address or roaming laptop users.
2. As of now, the VPN must be policy based. I cannot make it work with a route based config, as the client sends its internal address for the source network in the proxy-id. The client is not very configurable.

The first thing we need to do is create certificates. You will need a machine certificate on your mac, and a VPN server certificate on your firewall. The CA cert will need to be loaded on each device, and you will have to place it in the System Keychain on the Mac to make it trusted. I created my own CA using the Certificate Assistant on the Mac.

1. Create the CA - Open Keychain Access, and under the Keychain menu at the top, go under certificate assistant and choose "Create a certificate Authority". Follow the prompts to get it created. Once it is created, drag a copy to the desktop, and then drag the file to your System keychain. If you drag it directly from one keychain to another, it will move it, not copy it. Make sure you choose a unique serial number when you get to that point.

2. Create the Machine certificate - User Certificate Assistant to create a new cert. Call it "VPN client Cert" or whatever you want. This is a Leaf certificate, not a self-signed. Select the box that lets you override options. The email address should be vpn@yourdomain.com, or whatever you plan on using for an IKE ID on the Juniper. Set a unique serial number. You should choose 1024-bit/RSA. Uncheck the "Critical" checkbox, and only check Signature and Key Encipherment. Include Extended Key Usage, uncheck critical, check SSL Client, SSL Server, PKINIT Client, and PKINIT Server. Uncheck Basic Contraints in the next screen, and uncheck Subject Alternate Name in the one after that. Select your System Keychain to store it.

3. Load the CA Cert on the Juniper. Create a CSR, and make sure the FQDN or the IP Address is set to *exactly* what the VPN client will point to. Save the CSR. Start the certificate assistant again and select the "create a certificate for someone else" option. Drag the CSR onto the area it tells you to. You must override the options for this one also. You need to select VPN Server, make sure the "Extended Key Usage" option is UNCHECKED. Include a Subject alternate name, and fill in either the DNS name or the IP. The other fields should be blank, only fill in one of them. Once the cert is created, you will need to load it on the Juniper.

4. You MUST create a CRL. Certificate assistant doesn't have an option for this. You'll have to use OpenSSL to do it on the command line. I don't have the instructions in front of me now, but you'll just want to create a blank CRL. Search google for now, I'll put up instructions when I get around to it.

(Note that there are bugs with the Keychain on both Leopard and Tiger. Google can help with some of them. But, on Leopard, I noticed if you delete *anything* from the keychain, you cannot create certificates again until you reboot. Also, it is imperative you choose unique serial numbers for all of your certs. If you do not, you can possibly corrupt the keychain, just ask me how I know.)

---- Configure the Mac (Leopard instructions)
1. Open Network under the System Prefs. Add a new interface, choose L2TP over IPSEC. Name it something. At the drop down menu at the top, you MUST change it from Default and add a new configuration called something else. If you do not, it will not take your machine cert... Bug?
2. Put in the hostname or IP that you used in the Subject Alternative Name section in your Juniper cert.
3. Put in the desired username.
4. Click on advanced and put in the password.
5. Under the Machine Authentication Section, select certificate, then select your machine cert. Click OK.
6. Click Apply.

You can either find the checkbox that says "Send all traffic over the VPN", or you can manually enter routes when you connect: route add -net 10.1.1.0/24 -interface ppp0

I'm researching DHCP option 121 to see if this can be used in conjunction with L2TP to push routes down to the client. Let me know if you find anything.

---- Configure the Juniper
These are fairly abbreviated instructions, but should get you started.

1. Configure a user. Select IKE User, Simple Identity, Auto, and for the IKE ID, put in "Email=vpn@yourdomain.com" or whatever email addy was put in your Machine cert
2. Select L2TP user
3. Click OK
4. Add that user to a group if there is going to be more than just one person using the VPN
5. Configure an IP pool
6. Configure your L2TP tunnel, choose the IP Pool you just created, and set up whatever DNS/WINS settings you want
7. Create a P1 proposal with RSA/DH Group 2/3DES/SHA-1/3600 sec lifetime
8. Create a P2 proposal with NO PFS/AES-128/SHA-1/3600 sec lifetime
9. Create an Autokey IKE gateway with your User or User Group, click advanced, Choose your P1 proposal, NAT Traversal if needed (likely), Your Juniper cert you created, the CA Cert you loaded, and PKCS7 (Might work with X509-sig also)
10. Press Return, then OK
11. Create a new Autokey IKE VPN. Choose your new gateway, click advanced, Choose your P2 proposal, Transport mode, Hit return, then OK.
12. Create an Untrust->Trust policy. Source is Dialup VPN, Destination is your internet networks. Action is Tunnel, choose your VPN and your L2TP tunnel. Position at Top. Click OK.

It should now work. Make sure you test from the outside. By far the worst part of this is the certificate stuff. The subjectAltName must be set in the cert loaded on the Juniper and the Extended Key Usage option must NOT be set, or the OSX client will throw it away. Also, you must ensure that the CA Cert is loaded into your System Keychain and marked as Trusted for all users, or it will not trust the cert given to you by the Juniper. The CRL must also be loaded on the Juniper, I couldn't find a way to make it not check that, even though there is an option to ignore it. You *could* use the same machine certificate for all of your clients if you really wanted to, because they are still authenticating with user/pass for L2TP. But, it's definitely better that every gets their own certificate. I'll say this again also, make sure you have unique serial numbers for all of your certs. I ended up having to delete my System keychain several times because I didn't pay attention and just assumed that the certificate assistant would autoincrement my serial numbers for me.

I hope this helps. There may be some unnecessary options checked in the machine cert, but those were the only ones I could check to get it to actually show up as a valid machine certificate on the Mac. Sorry for the less than detailed instructions, but I wanted to get something up there while I had some time. If anyone wants to take some screenshots during the process and send them to me, I'll whack up a better Howto and toss them in there.