Analyzing Text and Generating Content with Neural Networks and TensorFlow

Can convolutional neural networks, typically used for image processing, accelerate text processing? Where do word embeddings come in here to help? How to generate unique content by using TensorFlow? This blog post explores these questions as discussed at the recent TensorFlow meetup in Denver.

Natural language processing with neural networks

Ville Kallioniemi, a software engineer at Oracle, focused on word embeddings as a means of natural language processing (NLP). He explained that word embeddings represent a basic building block of NPL systems, which need numerical inputs. As an example, he mentioned a popular machine learning algorithm—word2vec. It helps to create embeddings using a shallow neural network that predicts the next word or surrounding words.

(Previously, we have briefly written about text analysis with word2vec, which can view a sequence of words as vectors.)

Though convolutional neural networks (CNN) are mostly used for image processing, Ville highlighted that one can get satisfying results when applying them for text classification. However, one has “to turn a word into something that a CNN will understand.” That’s where word embeddings come in to split text into “eatable” pieces.

Ville also exemplified a few scenarios where text classification can be of use:

• e-mail (ham or spam)
• tweet sentiment (positive, negative, or neutral)
• social media posting (a topic company X is interested in)

You can find Ville’s presentation below.

Generating content with TensorFlow

In his session, Martyn Garcia of 255 BITS overviewed a number of tools that emerge from TensorFlow and help to generate unique content. Explaining how these tools work, he divided them into two groups:

• Sequence: char-rnn, sketch-rnn, and wavenet
• Real-valued: autoencoders, generative adversarial networks, vae, and hybrid

Take a look at Martyn’s presentation here.

Join our group to stay tuned with the upcoming events.

Want details? Watch the video!

Further reading

• Using Convolutional Neural Networks and TensorFlow for Image Classification
• Image and Text Recognition with TensorFlow Using Convolutional Neural Networks
• Text Prediction with TensorFlow and Long Short-Term Memory—in Six Steps
• How TensorFlow Can Help to Perform Natural Language Processing Tasks

About the experts

Ville Kallioniemi is Software Engineer at Oracle. He has a degree in computer engineering, embedded systems, and telecommunications from Åbo Akademi University, as well as took courses in machine learning and artificial neural networks. Ville has experience in developing deep learning solutions, electronic health record systems, and applications for adaptive learning.

Martyn Garcia is Full-Stack Engineer and a cofounder of 255 BITS. He holds a bachelor’s degree in computer science from Texas Tech University. Martyn has experience in developing software for information technology, finance, and retail companies. His professional interests lie within consulting, machine learning, and open-source projects, such as TensorFlow.