There are a lot of kayak and canoe rentals on the region, typically associated with good places to go. I've listed a few, mainly just to test software. If you have a store or business and you'd like to be listed here, contact me.
The Walkill is the only river that originates in New Jersey and flows out - north over the border with New York, eventually draining into the Hudson. So it gets out, but it comes back.