## More Bézier Walks in Neural Networks

The videos above were generated using the same script described in an earlier post.

## Random Bézier Walk in a Random Neural Network

The video above was generated using neuralart.

## Neural Art

neuralart is a Python library and utility for rendering generative art from a randomly initialized neural network.

It’s based on the following blog posts and pages from studio otoro.

The package is available on PyPI, the Python Package Index. It can be installed with pip.

$pip install neuralart For example command line usage, see neuralart#example. For example library usage, see example.py. ## echo and printenv in x86 Assembly This post contains implementations of echo and printenv in 32-bit x86 assembly for Linux. echo is a Unix utility that prints its arguments to standard output. printenv is a Unix utility that prints the environment to standard output. The core functionality of these programs can be written in a few lines of C, where program arguments and the environment are passed as function arguments to main. When a process is executed on Linux (or other Unix-like systems), its stack contains pointers to the program arguments and the environment, as shown below.  |--------------------------| Low 0(%esp) | Argument Count | Addresses |--------------------------| 4(%esp) | Argument Pointers | | ... | |--------------------------| | 0 | |--------------------------| | Environment Pointers | | ... | |--------------------------| | 0 | |--------------------------| | Additional Data | | ... | High |--------------------------| Addresses  Tagged , , , , | Leave a comment ## k-means Image Color Quantization I implemented a web page that can apply color quantization to images using k-means clustering. Here’s the link: https://dstein64.github.io/k-means-quantization-js/ The JavaScript source code is available on GitHub: https://github.com/dstein64/k-means-quantization-js ## Factorization Machines with Theano A Factorization Machine (FM) is a predictive model that can be used for regression and classification (Rendle 2010). FMs efficiently incorporate pairwise interactions by using factorized parameters. PyFactorizationMachines is a Theano-based Python implementation of factorization machines. Update 4/20/2017: The library is now available on PyPI, the Python Package Index. It can be installed with pip. $ pip install pyfms

For documentation, see documentation.md.

For example usage, see example.py.

## Conway’s Game of Life

Here’s a quick-and-dirty implementation of Conway’s Game of Life.

Cells can be selected/deselected by clicking and dragging your mouse. The interface and display were designed for use with a desktop/laptop computer, not a touchscreen mobile device. That is, cells can’t be selected by swiping (but tapping works).

The default selected cells spell my first name, daniel. After selecting cells, click Start to begin the game of life.

The source code is available here (use your browser’s view source).

## Matrix Factorization with Theano

Matrix factorization algorithms factorize a matrix D into two matrices P and Q, such that D ≈ PQ. By limiting the dimensionality of P and Q, PQ provides a low-rank approximation of D. While singular value decomposition (SVD) can also be used for this same task, the matrix factorization algorithms considered in this post accommodate missing data in matrix D, unlike SVD.

For an overview of matrix factorization, I recommend Albert Au Yeung’s tutorial. That post describes matrix factorization, motivates the problem with a ratings prediction task, derives the gradients used by stochastic gradient descent, and implements the algorithm in Python.

For exploratory work, it would be great if the algorithm could be implemented in such a way that the gradients could be automatically derived. With such an approach, gradients would not have to be re-derived when e.g., a change is made to the loss function (either the error term and/or the regularization term). In general, automatically derived gradients for machine learning problems facilitate increased exploration of ideas by removing a time-consuming step.

Theano is a Python library that allows users to specify their problem symbolically using NumPy-based syntax. The expressions are compiled to run efficiently on actual data. Theano’s webpage provides documentation and a tutorial.

The following code includes a Theano-based implementation of matrix factorization using batch gradient descent. The parameters are similar to those in the quuxlabs matrix factorization implementation. D is a second-order masked numpy.ndarray (e.g., a ratings matrix, where the mask indicates missing ratings), and P and Q are the initial matrix factors. The elements of P and Q are the parameters of the model, which are initialized by the function’s caller. The rank of the factorization is specified by the dimensions of P and Q. For a rank-k factorization, P must be $m \times k$ and Q must be $k \times n$ (where D is an $m \times n$ matrix). Additional parameters specify the number of iterations, the learning rate, and the regularization importance.

The code doesn’t contain any derived gradients. It specifies the loss function and the parameters that the loss function will be minimized with respect to. Theano figures out the rest!

## Article Highlighter

Auto Highlight is a Chrome extension that automatically highlights the important content on article pages.

Here’s a link to the extension:

The source code is on GitHub:
https://github.com/dstein64/highlight

After installing the extension, a highlighter icon appears in the location bar. Clicking that icon highlights important content on the page.

## Anchor Graph Hashing in Python

I was recently collaborating on a project that relied on hashing (where I’m referring to “hashing” the same way it’s used in locality sensitive hashing, as opposed to its more conventional usage). One of my contributions was an implementation of Anchor Graph Hashing (AGH) [1] in Python. The code was integrated into the project, but I’ve uploaded the AGH module to its own separate GitHub repository.

https://github.com/dstein64/PyAnchorGraphHasher

[1] Liu, Wei, Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. 2011. “Hashing with Graphs.” In Proceedings of the 28th International Conference on Machine Learning (ICML-11), edited by Lise Getoor and Tobias Scheffer, 1–8. ICML ’11. New York, NY, USA: ACM.