{"id":25929,"date":"2019-07-11T00:39:29","date_gmt":"2019-07-11T04:39:29","guid":{"rendered":"https:\/\/www.dannyadam.com\/blog\/?p=25929"},"modified":"2020-01-14T22:09:10","modified_gmt":"2020-01-15T03:09:10","slug":"compressing-vgg-for-style-transfer","status":"publish","type":"post","link":"https:\/\/www.dannyadam.com\/blog\/2019\/07\/compressing-vgg-for-style-transfer\/","title":{"rendered":"Compressing VGG for Style Transfer"},"content":{"rendered":"\n\n \n \n \n \n \n \n
\n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n <\/tr>\n
32-bit float (no quantization)<\/td>\n 8-bit<\/td>\n 7-bit<\/td>\n <\/tr>\n
\n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n <\/tr>\n
6-bit<\/td>\n 5-bit<\/td>\n 4-bit<\/td>\n <\/tr>\n
\n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n <\/tr>\n
3-bit<\/td>\n 2-bit<\/td>\n 1-bit<\/td>\n <\/tr>\n<\/table>\n\n\n\n

I recently implemented pastiche<\/a>\u2014discussed in a prior post<\/a>\u2014for applying neural style transfer. I encountered a size limit when uploading the library to PyPI<\/a>, as a package cannot exceed 60MB. The 32-bit floating point weights for the underlying VGG model [1<\/a>] were contained in an 80MB file. My package was subsequently approved for a size limit increase that could accommodate the VGG weights as-is, but I was still interested in compressing the model.<\/p>\n\n\n\n

Various techniques have been proposed for compressing neural networks\u2014including distillation [2<\/a>] and quantization [3<\/a><\/a>,4<\/a><\/a>]\u2014which have been shown to work well in the context of classification. My problem was in the context of style transfer, so I was not sure how model compression would impact the results.<\/p>\n\n\n\n

Experiments<\/span><\/strong><\/h4>\n\n\n\n

I decided to experiment with weight quantization, using a scheme where I could store the quantized weights on disk, and then uncompress the weights to full 32-bit floats at runtime. This quantization scheme would allow me to continue using my existing code after the model is loaded. I am not targeting environments where memory is a constraint, so I was not particularly interested in approaches that would also reduce the model footprint at runtime. I used kmeans1d<\/a>\u2014discussed in a prior post<\/a>\u2014for quantizing each layer’s weights.<\/p>\n\n\n

<\/p>\n\n\n

Before I implemented support for loading a quantized VGG model, I first ran experiments to see how different levels of compression would impact style transfer. I did not conduct extensive experiments\u2014just a few style transfers at different levels of compression. quantize.py<\/a><\/code> creates updated VGG models with simulated quantization, and quantized_pastiche.sh<\/a><\/code> runs style transfer using the updated VGG models. These scripts are in a separate branch<\/a> I created for the experiments.<\/p>\n\n\n\n

The images at the top of this post were generated with Edvard Munch’s The Scream<\/a> and a photo<\/a> I took at the Pittsburgh Zoo in 2017. The images below were generated with Vincent van Gogh\u2019s The Starry Night<\/a><\/em> and a photo<\/a> I took in Boston in 2015. The image captions indicate the compression rate of the VGG model used for the corresponding style transfer.<\/p>\n\n\n\n\n \n \n \n \n \n \n
\n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n <\/tr>\n
32-bit float (no quantization)<\/td>\n 8-bit<\/td>\n 7-bit<\/td>\n <\/tr>\n
\n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n <\/tr>\n
6-bit<\/td>\n 5-bit<\/td>\n 4-bit<\/td>\n <\/tr>\n
\n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n \n \n \n <\/a>\n <\/td>\n <\/tr>\n
3-bit<\/td>\n 2-bit<\/td>\n 1-bit<\/td>\n <\/tr>\n<\/table>\n\n\n\n

Implementation<\/span><\/strong><\/h4>\n\n\n\n

I originally decided to compress the model using 6-bit weights, and ran a few additional style transfers to check the quality at this compression level. I modified<\/a> the code to generate and load VGG models with weights quantized to arbitrary bit widths. Unfortunately, my implementation had a noticeable effect on latency when loading the model, taking almost twenty seconds for a model with weights compressed to 2 bits (I didn’t test for other compression rates, but larger bit widths would presumably take longer).<\/p>\n\n\n\n

I subsequently decided to quantize the weights to 8 bits instead of 6 bits, since this allowed for fast processing using PyTorch’s built-in uint8<\/code> type. The VGG file size decreased from 80MB to 20MB, well within the 60MB PyPI limit that I originally encountered. Loading the quantized model takes less than 1 second.<\/p>\n\n\n\n

References<\/span><\/strong><\/h4>\n\n\n\n

[1] Simonyan, Karen, and Andrew Zisserman. \u201cVery Deep Convolutional Networks for Large-Scale Image Recognition.\u201d ArXiv:1409.1556 [Cs], September 4, 2014. http:\/\/arxiv.org\/abs\/1409.1556<\/a>.

[2] Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. \u201cDistilling the Knowledge in a Neural Network.\u201d ArXiv:1503.02531 [Cs, Stat], March 9, 2015. http:\/\/arxiv.org\/abs\/1503.02531<\/a>.

[3] Vanhoucke, Vincent, Andrew Senior, and Mark Z. Mao. \u201cImproving the Speed of Neural Networks on CPUs.\u201d In Deep Learning and Unsupervised Feature Learning Workshop, NIPS 2011.

[4] Han, Song, Huizi Mao, and William J. Dally. \u201cDeep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding.\u201d ArXiv:1510.00149 [Cs], October 1, 2015. http:\/\/arxiv.org\/abs\/1510.00149<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

32-bit float (no quantization) 8-bit 7-bit 6-bit 5-bit 4-bit 3-bit 2-bit 1-bit I recently implemented pastiche\u2014discussed in a prior post\u2014for applying neural style transfer. I encountered a size limit when uploading the library to PyPI, as a package cannot exceed 60MB. The 32-bit floating point weights for the underlying VGG model [1] were contained in […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[46,56,65,72,71],"class_list":["post-25929","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-machine-learning","tag-neural-networks","tag-neural-style-transfer","tag-quantization","tag-vgg"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p1sCC6-6Kd","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/posts\/25929","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/comments?post=25929"}],"version-history":[{"count":104,"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/posts\/25929\/revisions"}],"predecessor-version":[{"id":26270,"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/posts\/25929\/revisions\/26270"}],"wp:attachment":[{"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/media?parent=25929"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/categories?post=25929"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dannyadam.com\/blog\/wp-json\/wp\/v2\/tags?post=25929"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}