Analyzing Text and Generating Content with Neural Networks and TensorFlow
Can convolutional neural networks, typically used for image processing, accelerate text processing? Where do word embeddings come in here to help? How to generate unique content by using TensorFlow? This blog post explores these questions as discussed at the recent TensorFlow meetup in Denver.
Natural language processing with neural networks
Ville Kallioniemi, a software engineer at Oracle, focused on word embeddings as a means of natural language processing (NLP). He explained that word embeddings represent a basic building block of NPL systems, which need numerical inputs. As an example, he mentioned a popular machine learning algorithm—word2vec. It helps to create embeddings using a shallow neural network that predicts the next word or surrounding words.
(Previously, we have briefly written about text analysis with word2vec, which can view a sequence of words as vectors.)
Though convolutional neural networks (CNN) are mostly used for image processing, Ville highlighted that one can get satisfying results when applying them for text classification. However, one has “to turn a word into something that a CNN will understand.” That’s where word embeddings come in to split text into “eatable” pieces.
Ville also exemplified a few scenarios where text classification can be of use:
- e-mail (ham or spam)
- Tweet sentiment (positive, negative, or neutral)
- social media posting (a topic company X is interested in)
You can find Ville’s presentation below. Watch the video at the bottom of this post.
Generating content with TensorFlow
In his session, Martyn Garcia of 255 BITS overviewed a number of tools that emerge from TensorFlow and help to generate unique content. Explaining how these tools work, he divided them into two groups:
- Sequence: char-rnn, sketch-rnn, and wavenet
- Real-valued: autoencoders, generative adversarial networks, vae, and hybrid
Take a look at Martyn’s presentation here. For more details, watch the video from the meetup.
Join our group to stay tuned with the upcoming events.
About the speakers
Ville Kallioniemi is a software engineer for Oracle, working on deep learning applications. His professional experience ranges from embedded systems to machine learning.
Martyn Garcia is a full stack engineer focusing on machine learning. When not consulting, he contributes to open-source projects, TensorFlow in particular.
To stay tuned with the latest updates, subscribe to our blog or follow @altoros.