Human bodies are undoubtedly fragile, and the most vulnerable part to permanent damage and function is the brain. While there are also other parts that can cause great pain, nothing can make you drop down with no consciousness like taking force to the chin.
The chin is so vulnerable that it doesn’t take much force to rock the brain. This means that skinny individuals generate more than enough force to rock a big dude. After all, being big does not mean you can take much more force to the chin. An example is Valuev vs Haye, where Valuev was a 331 pound 7 foot monster getting rocked by a left hook.