Can you use non-fat milk in place of whole milk in a panna cotta recipe?!

I have a recipe for Panna Cotta with fresh berries, it specifically calls for 1 cup of whole milk. Unfortunately in my house we don't drink whole milk only non-fat. I went to the store to get one of those small containers of milk & good old Safeway half had large containers of milk which would be a waste in my household. I'm thinking it shouldn't make a difference since it sets with unflavored gelatin.

Any thoughts?

Thanks!