Um, Sunday evening - a train from Penn Station to Newark airport about every 7 minutes. Taxi is more than from JFK but when the train is $12.50, frequent and takes maybe 20 minutes a taxi is silly. Maybe you hit a bad train day but we find Newark far more convenient than JFK.

On weekdays, between 6 a.m. and 9 p.m., at least six NJ TRANSIT trains arrive every hour at the airport station. Between 9 p.m. and midnight, four trains arrive every hour
http://www.njtransit.com/rg/rg_servlet.srv?hdnPageAction=AirportConnectionsTo

The timetables are appallingly confusing on paper, took me ages to work out

I like the Accela, but we're coming from either London or SF ;-)