[AI] GPT2 모델 사용해보기

AI에 대해 학습되지않은 상태로 깡으로 머리박았을 시점에 작성한 글임
(아무런 영양가 없음)

1.GPT2란

• GPT 모델중 하나로 최대 15억 개의 파라미터를 가지고있다.(참고로 GPT-3가 1750억개) 대신 직접 모델 파일을 다운받아서 사용할수 있기 때문에 이번기회에 한번 다운받아서 사용해 보았다.

 

 

2.깃헙 코드

• Openai의 gpt-2 돌리기 위한 코드는 다음에서 clone 하여 사용하였다.

https://github.com/openai/gpt-2

GitHub - openai/gpt-2: Code for the paper "Language Models are Unsupervised Multitask Learners"

 

 

 

3.세팅

• 내부에 Docker를 위한 파일들도 준비 되어있지만 나는 venv 를 만들고 환경을 구축하기로하였다.

 

 

3-1.파이썬 버전 3.7

• 이게 조금 오래된 모델이라 tensorflow를 1.15 버전을 사용해야하는데 이버전이 파이썬 높은 버전에서는 또 지원을 안해서 어쩔수없이 3.7버전을 다운받아서 가상환경으로 만들었다.

python -m venv venv //3.7버전으로만들었음

• requirements.txt가 있길래 이거랑 tensorflow 1.15 버전을 환경에 적용시켰다.

pip install -r requirements.txt
pip install tensorflow==1.15

 

 

3-2 모델다운로드

• clone 한 파일중에 download_model.py로 모델을 다운받으면 되는것같다. 나는 774M 모델로 받았다.

import os
import sys
import requests
from tqdm import tqdm

if len(sys.argv) != 2:
    print('You must enter the model name as a parameter, e.g.: download_model.py 124M')
    sys.exit(1)

model = sys.argv[1]

subdir = os.path.join('models', model)
if not os.path.exists(subdir):
    os.makedirs(subdir)
subdir = subdir.replace('\\','/') # needed for Windows

for filename in ['checkpoint','encoder.json','hparams.json','model.ckpt.data-00000-of-00001', 'model.ckpt.index', 'model.ckpt.meta', 'vocab.bpe']:

    r = requests.get("https://openaipublic.blob.core.windows.net/gpt-2/" + subdir + "/" + filename, stream=True)

    with open(os.path.join(subdir, filename), 'wb') as f:
        file_size = int(r.headers["content-length"])
        chunk_size = 1000
        with tqdm(ncols=100, desc="Fetching " + filename, total=file_size, unit_scale=True) as pbar:
            # 1k for chunk_size, since Ethernet packet size is around 1500 bytes
            for chunk in r.iter_content(chunk_size=chunk_size):
                f.write(chunk)
                pbar.update(chunk_size)

python download_model.py 774M

 

 

 

 

3-3. 모델 실행

• src에 interactive_conditional_samples.py 파일에 내가 추가한 모델 명을 적은 다음에 실행해 보았다.

def interact_model(
    model_name='774M',
    seed=None,
    nsamples=1,
    batch_size=1,
    length=None,
    temperature=1,
    top_k=0,
    top_p=1,
    models_dir='models',
):

python src/interactive_conditional_samples.py

터미널에 다음처음 프롬프트를 입력해달라고 한다. 한번 입력해보자

Instructions for updating:
Use `tf.random.categorical` instead.
WARNING:tensorflow:From src/interactive_conditional_samples.py:68: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

Model prompt >>>  

 

 

 

 

입력1

• 친절한 말투로 5문장으로 웹개발에 필요한 기술스텍을 정리해달라고 했다.

Summarize the basic tech stack needed for web development in 5 sentences with a friendly tone

결과2

• 결과가 나오기까지 꽤나 오랜시간이 걸렸다.(쇼츠 2개정도) 근데 결과는 그에 비해 좋지않은 결과가 나왔다. 아무래도 모델이 구식이다 보니 이런 짧은 프롬프트로는 잘안되는듯하다.
• 웹개발 관련 내용이 나오긴했지만 쓸모없는 내용을 많이 담고있다.

======================================== SAMPLE 1 ========================================
.

Our full-stack Javascript framework breaks down each code block into high-level concepts.

You'll feel like a web developer, from the first line of code. Where will this code go? What's it doing?

Focuses on building high-level concepts: how to create an HTML file, how to get the HTML going, JavaScript (JavaScript does it all), CSS, and UIs.

We'll cover topics that will freshen up your current Javascript or hosted development skills while you improve your JavaScript skills:

Fast improvements in knee jerk reaction coding also help to give yourself more time for experimenting and breaking out of your comfort zone.

Learn from having a solid base of knowledge won't get you canned by your boss when you publish (or…).

Build a high-throughput web application with a dedicated database, deep service layers, an easy-to-customize frontend, and an integrated development lifecycle.

How to write asynchronous code to build distributed applications, load balance, or backend services.

Know Social Media and Online Marketing Terms Understand the terminology and concepts used by companies like Facebook, Twitter, and Linkedin to help make their products accessible and attractive to users. More… Meets Microsoft's Acceptable Use Policy Open Source, Proposed and Proposals & Contests This will be at the back end of coding classes for Web Developers. Why? For those who want to be ready for the challenges of creating an interesting and challenging project. Especially worth the coursework if you didn't get into the programming field in college.

Presolve your funding needs with go to market.

Evaluate résumés, cover letters, and cover dates from the author.

Track SEO performance Trends, and repay fees and insurance with percentages based on the real in-site effect. Practice Code Crunch and Visual Career Design in 15 Minutes Improve faster and stay more successful by identifying and fixing web coding mistakes faster.Learn whether coding bugs complicated projects or why our solutions completely suck in 1 hour.A coding challenge based on JavaScript and incumbent firms' website optimization to build web applications – even if done in a labyrinthine code base like WordPress or Drupal.

Specifying killing two birds with one stone with a project that values team communication over self-esteem.

Career Advice The screen show will coincide with one of the following fixed dates: The @Trailofdisaster Monster Cubs 2014 are at First Annual Spring Training on Tuesday

 

 

 

 

입력2

• 이번엔 그러면 한번 길고 자세하게 프롬프트를 짜보자

"Please summarize the basic tech stack needed for web development
in 5 sentences.Each sentence should focus on one key concept,
without going into too much detail.Avoid adding examples
or overly specific explanations; just mention the essential
technologies.Write in a friendly and easy-to-understand tone
.Include key web development technologies such as HTML,CSS, JavaScript, frontend,
and backend.Do not include tips orpractical examples, just the technical concepts."

결과2

• 여전히 이상한 이야기를 말한다.

Capable HTML 5 client and server code and CSS view formulas E-commerce grid 
styles business cards Web analytics technical transactions Get the Edge brow
ser<|endoftext|>Home Events Calendar Donations Type Newspaper Bus Stamps Gr
oup Giving Kids Projects Moneys to Give Children's Movers Puppy fosters Tra
in volunteer Volunteer services City Catalog Community Services Business CS
A Community rights Directors Grants & gifts Grants Call 2018 NEWSLETTER 201
8 A TON VOLUNTEER AT SITE Annual Thanksgiving and Christmas Dinner A month
to typical SBE Annual Be silent, be careful, be merry – Share the Lord ja
r 2015 BE SILENT, BE THIN Staff Part-Time Bikes Open House Biking on the 
Madison House Estate Barber BOY STEALING 2017 (Fill in as best you can) B
idding of Speaker Through it all, remember that the only constant in life
is change. #tornadocayilton Women's Sampler Female Hockey Lunch Break 
Flower power Fit and live out of your Hepworth cooler Help get these 
lovely Kemper Tudor amphibious bikes on the road in 2018 by sharing 
on car-loving on-street Clothes Curbside when you see one Donate 
some cookies to the Austin Community Food Bank Donate to the Retirement
Community Home or the McQuinn Aging Center and Estate Fund for Austin
Donate eggs to Austin 106 Toy volunteer weather pilot 2014 Franciscan
Conference 2014 We are always here, wherever Grace, the Lady of Graceville,
is. Join with the women in ministry brief military wedding on October 
12 2017 Ages don't let age limit you! Writers Rally in vigil at Glados
Beat the heat & dark with better breath and definitely turn your mind 
off to what's out (hard or soft) Austin Events Calendar Events for Christmas
Eve, Labor Day, Epiphany, and Thanksgiving September 27, 2018 11, 11:30,
12, 1-2 PM, 1-8 PM ON THE DETAINEES FINDERS DISCONTINUED October 6, 2018 2,
3, 4, 8, 9, 12, 3-5:30, 5-7 AM, 10:30 AM-12:30 PM, 1-5:30 PM GATEWAY BUILDERS:
AWARDED FUNDS 2018 Categories: Arts and Crafts, Community Service, Food, 
Hobbies, Family Activities, Garden, Glades, Hen House Arboretum, Lindenhurst 
Garden Foundation, DNR, Natural Areas, Resources, Young/Old Midtown Writers,
Anchor Anchor, August Wilson and Mearthen County

 

 

 

 

 

입력3

• 이번엔 gpt 4.5 에게 프롬프트를 고쳐달라고 하고 넣어봤다.

Please summarize the essential technologies needed for web development in 5
concise sentences. Each sentence should focus on one key technology or concept,
and avoid going into too much detail or offering practical examples.
Do not include unrelated topics or tips. The summary should mention 
the following core technologies: HTML, CSS, JavaScript, frontend, 
and backend. Write in a friendly and easy-to-understand tone, making 
sure the explanation is suitable for beginners who want to get an overview 
of web development.

결과3

• 조금 더 나아진거같기도 하고 여전히 이상하다

======================================== SAMPLE 1 ========================================
 Examples of appropriate use would be given in the summary. (This recognition should not be considered an endorsement.) Have a good idea of what corresponds to a core technology. For website building platforms, "portability" is generally defined as being able to use the same technology on several business sites. Hear from companies, professions, and organizations with specific use cases, ideally including descriptions of how many people need/do it, when it's needed most, and how it's done. Explain specific procedures that were necessary for presentation. Sample use cases might be found on the Web Portal's Brendan's List podcast series.

Be Beginner-Friendly Do not use technical terminology you have never used before, or completely new technologies.

Stick with simple tools and justifications when appropriate.

Answer all questions as realistically as possible.

Don't focus on techniques you have never tried.

For assistance, consider StackOverflow, the website of technology advice community Improving Careers Together, or visiting StackOverflow.com or landing page How *Do* I Put this link on my website for this person's article?. 

Using your recordings In addition to the Informatica Skill Seeker webinar series, Informatica Education now offers audiovisual aids in the following form: Infogalactic Sound work Fear Fatigue Research ...............................................................................................................................................subtopics -__- -__- -__- -__- -__- -__- -__- -__- The Basics of ___________. Except for enhancement of fundamental practices, Networks can be used everywhere since 1 November 2008. SOURCE Information, and other recommendations in the Informatica Skills Seeker series

GMs living in the region for the next 3 months

Foster mum and dad

Try it out with your friends

Covering networks will be useful

Close innovation pitches for employment with industry

"Wave your person through a lower tier office with great mobility if there is an opportunity for you to contribute an in-depth technical solution to their work." Summary of the concepts

Ten important use cases that organisations consider as consequences

Semantic web

Increasing accessibility to computing devices

Building great 15-minute assumptions

Exploring industry tell-ivations and cases

AI - An Application When it comes to GMs, now is the time to make it your success-partner. Do you ever visit anyone's GP? I dare you to do it