Nowadays kayaks are commodity items like bicycles, and it is easy to buy a thing that you have no experience in using. So here are some pointers for beginners.
Parking and launch access are free. There can be a good deal of boat traffic on the river, especially ferries, but if you are careful, you can paddle around the Statue of Liberty and Ellis Island.