Haiku Generator

None

Haiku Generator

Create AI generated haikus!

Product Overview:

The Haiku Generator creates a fun haiku using NLP (in this case a specified GPT-2 model). You can prompt the AI with the first word of the haiku, and the AI will try finish it for you or suggest another

Why this product?

The Haiku Generator is fun and easy to use! If you have writer's block or if you just want to read a short poem, the haiku generator has you covered

Our Motivations

1

We wanted to do something with text generation

2

We like poetry :D

3

Haikus are short and stick to a strict outline

4

Haikus are fun! And it's interesting to see a machine create art

Resources

Some things that helped us develop our model

Data
  • Haikuzao Dataset

    We used the Haikuzao dataset to train our AI model. However, during the data preprocessing, after condensing each poem into string list items, we had to clean some of the poems due to them being inappropriate.

Our AI model
  • We used GPT-2 by OpenAI for our model! GPT is a deep learning language model that can generate all types of language, from poems to stories that replicate human-like language, by using token to token prediction.

A Quick Overview of

GPT-2

Prediction

GPT is a language model that specializes in predicting the next word based on previous context, using masked self attention. (Other models, such as BERT use self attention which allows "fill in the blank " situations.)

Masked Self Attention

Masked self attention prevents the model from seeing tokens to the right of the position being calculated. (in the image above, it is at position 4)

Layers & Heads

A layer is a decoder, each layer has 12 different mechanisms (heads), for a total of 144 (12x12) attention patterns. These help the AI distribute importance between the relationships of individual words in a sentence.

1

Picking the function of our generator! We knew we wanted to do something in line with text generation. We ended up choosing to create a haiku generator.

2

Gathering data! We found multiple kaggle datasets but ended up using the Haikuzao dataset on github.

3

Understanding GPT (Generative Pre Training), the core of our model.

4

Training our model on the Haikuzao dataset, modifying settings to adjust the poems.

5

Cleaned some of the haikus in the dataset.

6

Combined the post-processing and the model, to create our final haiku generator.

7

Design a website to house our product and express our ambition.

8

share!

Difficulties Encountered

1

Errors and mistakes in code / debugging

2

cocalcg8 server being slow, and only tolerating one person running the code at once

3

Cocalc3 being slow

4

AI is trilingual and some of the haikus in the dataset were not pg-13

Meet The Creators!