Paddles
Your new boat probably came with a paddle. I've never seen a boat come with a good paddle. It is just a starter, to get you going. Get a good paddle that is the right size for you. It will make a world of difference. Keep the old one for a spare. Rule of thumb: your paddle should be about your height. If you install a thick seat cushion, you might want to go a little longer even.
I regard my paddle as an emergency signalling device as well, which is why I like the yellow color pictured, especially if your boat is some camouflage shade of green. The paddle above is about $50 at Dick's. I really like it. No sense hobbling yourself with the piece of junk that came with the boat.
The larger Dick's stores have a nice selection of paddles in all sizes up to 230cm. $50 will get a good one. Unless you are pretty short, just get 230cm.
Also, get some foam grips. Repetitive motion with wet hands is a sure way to get blisters, and you will quickly fall out of love with your new hobby. The grips above are quite nice and easy to install. Just spray a little soapy water inside, and they will slide right on. When they dry, they become fairly resistant to sliding, but are still adjustable. If you kayak a lot, you can go through a set of grips in a season. At about $9/pair on Amazoon, lay in a supply of these.
Finally, if you are going to go out alone, I recommend securing the paddle to the boat. Your paddle floats, but if it floats away and you don't have a friend to retrieve it, you are going swimming. If I'm not alone, I often disconnect it, although it's really not a bother. You can find pre-made lanyards, your paddle might even come with one nowadays. I like the lanyard to stay connected to the boat and disconnect from the paddle, and every lanyard you can buy is the opposite, so I just make my own.
Above is a six-pack of velcro clips that are prefect for kayak lanyards. The rest is just a few feet of 3/16" bungie cord with a loop at each end. The bottom loop attaches to any handy spot in the boat, typically a drain plug in front of the seat, and I make it extra-large so I can clip other things to it. I like to make sure everything is clipped to the boat, so if worst comes to worst you aren't chasing stuff all over the lake! And not everything floats. Again, you can find all this stuff cheap on Amazoon.
This paddle is nearing the end of its life, so I decided to try an experiment. I added a "drip notch". This is a small sharp-edged notch that the water flow can't follow. Instead it departs and falls well outside the boat. I hardly got a drop of water on me. To make the notch, simply drill a 1/4" hole by the edge of the plastic blade, 6-7 inches from the base. Then file it to the shape you see here, with a sharp corner at the edge. The back-cut assures the the stream can't jump over the notch to the other side. I can't claim this idea, I've seen a few paddles with this feature.
Update:
I added a second set of notches about 4 inches out from the first, and now it works even better. Just a few drops fall straight down from the inner notches.
Here is a useful option - and adjustable-length paddle. This makes a great spare or 'guest' paddle, as it will fit almost any situation. This paddle is not quite as sturdy and well-made * as the one at the top of the page, but is perfectly good for general use. The end shafts are aluminum, and the center piece is fiberglass, making for three joints. You can find these on Amazoon for around $40.
* How can I tell? Both of these paddles have sturdy aluminum shafts. However, the blades on the yellow paddle end in plugs that are glued inside the metal shaft, which is much stronger than the plastic and reinforces it. The blades on the adjustable paddle end in sockets that the shaft fits into, retained by a rivet or bolt. This is clearly not as strong, it has high-stress points, and someday the plastic socket will break, although maybe not in your lifetime. The adjustable paddle is also a little less stiff because it has two joints where the other has just one. Again, not a real problem.
Some people will try to sell you an expensive paddle with a carbon-fiber shaft. Carbon-fiber is not much stronger than aluminum, not much lighter, and a lot less tough. I would not waste my money.
Spares
Paddling with half a paddle is not nearly as bad as you would think - it is basically canoeing. It is slower and clumsier, but you can get home on a broken paddle. However, paddling with no paddle is a different story. For longer trips or going far from shore a spare paddle would be an excellent thing to have. If you are in a group, one spare paddle should suffice for all. I almost always keep my paddle tied to the boat so it can't get away.