Thanks for the info Flinkly, I figured it would work, but I just hadn't gotten around to doing it.
The nuts are a simple #6 push on nut, so you should be able to find those. ACE has a better selection of single nuts and assorted things than HD or Lowes, but really any hardware store should have something that will work. It might require a washer though.
I will find out when I do mine, but I think that "Foam" stuff is a polyurethane foam and will dissolve with the proper agent. This will probably also dissolve the paint. Body repair people can help us in this category. For sealing the hole again, pay close attention to the temperature range of your silicon, and caulk would definately not be a good idea. A urethane or polyurethane sealer would be best. If you're afraid that you might want to remove it again some day, there are various types of weatherstrip available at hardware stores with or without adhesive.