"Malware, short for malicious software, is any software used to disrupt computer operation, gather sensitive information, or gain access to private computer systems...a variety of forms of hostile or intrusive software."
http://en.wikipedia.org/wiki/Malware
By conscious choice, and for my own reasons, I am neither a fan nor a user of JetPack. However, I do still believe "malware" is neither fair nor accurate when describing it any more than it would be fair or accurate to call any of many completely-free plugins "malware" just because a given author has assumed or presumed something that is ultimately not liked or appreciated by each and every potential user.
https://wordpress.org/plugins/slimjetpack/ is one plugin making it possible for users to make at least partial use of JetPack without ever having to activate it, and there are other plugins also that do similarly:
https://wordpress.org/plugins/carousel-without-jetpack/
https://wordpress.org/plugins/jetpack-lite/
https://wordpress.org/plugins/tiled-gallery-carousel-without-jetpack/
https://wordpress.org/plugins/manual-control/
https://wordpress.org/plugins/slimjetpack/