Yes, their is drawbacks when not eating and drinking.
You have the bars in the lower left hand corner Energy,Food,Health,Comfort. If you do not eat or drink you energy,food and comfort will go down will effect the ability of the character to handle the fish and how long it takes you bring in the fish.
The Health bar is when you drink alcohol it will good down this will show how drunk you are. This has advantage and disadvantage when drinking alcohol such as when drink alcohol it will give extra XP when bring in the fish, but when you drink to much alcohol and the bar is all the way down you are drunk and it will effect how you handle the fish .
Your Comfort bar is how the character is feeling the way to bring that is food and drink just like your energy bar. If these both bars are down it will effect the characters ability of handling the fish and casting.
There are certain food and drinks that will bring your both bars up such as Honey, Coke, Tea,Coffee or and thing that has sugar, caffeine that will bring up your energy and comfort.
What will bring up your health bar if you drink to much alcohol is sour cream or over time it will slowly move up. But you have to remember to much alcohol will affect how you handle the fish that will a high chance of losing the fish or breaking your gear.
I hope this answer your question about the drawbacks when not eating and drinking.