[논문] Multi-persona LLM
[논문] Multi-persona LLM, UNLEASHING COGNITIVE SYNERGY IN LARGE LANGUAGE MODELS: A TASK SOLVING AGENT THROUGH MULTI-PERSONA SELF-COLLABORATION
Aug 19, 2023
Contents
Abstraction
혼자보다 여럿이 생각해서 내는 결과가 더 좋다.
LLM이 general한 문제해결 agent로 좋은 성능을 내고 있지만, 여전히 깊은 도메인 지식이나 복잡한 reasonin이 필요한 영역에 대해서는 잘 하지 못한다.
본 논문에서는 하나의 LLM(cognitive synergist)을 multi persona + multi-turn 콜라보레이션 시킨다(Solo Performance Prompting SPP).
한개 혹은 고정된 숫자의 페르소나보다 multiple, fine-grained LLM을 dynamic하게 결정했을 때 성능이 더 좋더라.
세가지 챌린지로 테스트해봤다. 1) Trivia Creative Writing, 2) Codenames Collaborative, 3) Logic Grid Puzzle. 지식이 필요한 작업과 리즈닝이 필요한 작업들이다.
바로 그림 몇개
![notion image](https://inblog.ai/_next/image?url=https%3A%2F%2Fwww.notion.so%2Fimage%2Fhttps%253A%252F%252Fs3-us-west-2.amazonaws.com%252Fsecure.notion-static.com%252F9cec51f6-64e5-471f-b8b1-bca5f6189461%252FScreen_Shot_2023-08-16_at_1.48.06_PM.png%3Ftable%3Dblock%26id%3Db378e8f2-134a-4070-897c-41a987767982%26cache%3Dv2&w=3840&q=75)
![notion image](https://inblog.ai/_next/image?url=https%3A%2F%2Fwww.notion.so%2Fimage%2Fhttps%253A%252F%252Fs3-us-west-2.amazonaws.com%252Fsecure.notion-static.com%252F1559cd46-e5ce-40d9-a14e-5028a62b9fbd%252FScreen_Shot_2023-08-16_at_1.50.11_PM.png%3Ftable%3Dblock%26id%3De945fe58-8ef9-4811-a46b-7be9992a4c6d%26cache%3Dv2&w=3840&q=75)
예시부터 보면, 우선 유저가 원하는 결과물이 있다. 포함되었는지 아닌지 explicit하게 파악이 가능한. 그냥 요구하면 3개의 요구 중 2개를 빼먹지만, agent하니까 낫더라.
Introduction
요약하면 하나의 LLM이 여러개의 multi persona로 분열된 후 셀프 협업을 통해서 지식, 리즈닝 양 측면에서 더 좋은 결과를 내게 하겠다.
그리고 이 LLM은 cognitive synergist라고 부르겠다.
위의 방법으로 했을 때 더 좋은 결과가 나올 것이라는 직관은 어디서 얻었는가? 1) the significance of pretend play and role-playing (The role of play in human development, 2009 논문), 2) 비슷하게 아이들의 발달에 관한 연구(더 오래된 - According to Piaget’s developmental theory (Piaget, 1954), engaging in pretend play and taking on different roles allows children to cultivate essential skills such as problem-solving, critical thinking, empathy, and cooperation)
그리고 앞서 일어났던 연구 중 특정한 페르소나를 LLM에게 부여하면 특정한 행동을 이끌어 낼 수 있다는 연구에서 영감을 얻었다. Instructing large language models to be distinguished experts라는 논문에서는 어떤 전문가의 페르소나를 부여했을 때 효과가 좋다는 연구를 보여준다.
여기서는 만약 영화 만들어라는 태스크에 → 영화 제작자, 10살 아이 이렇게 다양한 페르소나를 만들어서 사용한다.
멀티 페르소나라는걸 통해서 추가적인 retrieval system 없이 도메인 지식을 부여할 수 있었다.
multi-turn을 통해서 추가적인 에이전트 없이 self-revision, self-feedback이 가능하다.
위 그림에서처럼 AI Assistant라는 리더 페르소나가 존재하고, 얘가 시작하고 종료한다. 모든 페르소나가 positive feedback을 주면 종료된다.
3가지 key contributions
- SPP라는, 다이나믹하게 페르소나 정하고 멀티 턴으로 알아서 콜라보레이션하는 novel approach를 제안한다.
- 외부 리소스 없이 knowledge acquisition과 reasoning abilities를 향상 시킬 수 있었다.
- 왜 수가 고정된, coarse-grained(넓은 범주의) 페르소나가 아니라 dynamic, fine-grained(좁은 범주의) 페르소나가 좋은지 깊이 분석했다.
SPP 설명
- 페르소나 선정 : 여러개의 특별한 페르소나를 가진 에이전트를 우선 정해야한다. 무슨 페로소나를 만들어서 협업시키지? AI assistant - 리더 페르소나 포함
- Beginning remarks : 각자 자신이 가진 지식에 기반해 어떻게 태스크에 접근하는게 좋을지 말한다.
- Multi-persona iterative Collaboration : 리더 페르소나가 처음에 질문을 던지고, 다른 참가자들이 말을 하고, 반복적으로 점점 답에 가까워진다.
설정
GPT4-32k API를 썼고, temparature와 top_p는 1.0을 썼다.
답이 필요한 질문 :
- 결과 체크는 어떻게 했지? - 시 작성 같은 경우
- 실제로 어떻게 좋아졌다는거지?
- 자세한 구현 방법
- 그럼 Multi-agent가 특히 잘 할 수 있는 부분은 뭐고, 단점은 뭘까?
- 그럼 여기서 중요한건 뭘까? 원래의 multi-agent와의 차이점
짧은 해리포터 소설을 작성하도록 하는데, N개(5 or 10개)의 질문에 대한 답변을 소설안에 포함하도록 요구한다. → 결과에 몇개의 질문에 답을 했는지 개수를 체크해서 평가
![notion image](https://inblog.ai/_next/image?url=https%3A%2F%2Fwww.notion.so%2Fimage%2Fhttps%253A%252F%252Fs3-us-west-2.amazonaws.com%252Fsecure.notion-static.com%252F285d7665-85e3-42a8-8146-e3c384694de5%252FScreen_Shot_2023-08-16_at_2.59.14_PM.png%3Ftable%3Dblock%26id%3Dd748e946-20c2-4c29-8cbf-3b6bd379cc54%26cache%3Dv2&w=3840&q=75)
Trivia QA에서 PG or PG-13 등급에 해당하는 대중명사 100개를 골라 그걸 포함하도록 요청하는 질문 세트를 GPT-4를 이용해서 만들었다.
평가 - 생성의 일관성(우리가 흔히 말하는 소설의 개연성 등)은 너무 주관적인 기준으로 평가된다. 그래서 우리는 일부러 정량적으로 평가하려고 위의 세트를 만들었다.
![notion image](https://inblog.ai/_next/image?url=https%3A%2F%2Fwww.notion.so%2Fimage%2Fhttps%253A%252F%252Fs3-us-west-2.amazonaws.com%252Fsecure.notion-static.com%252Ff44bb383-c236-4897-9c7d-37e9cb369e7b%252FUntitled.png%3Ftable%3Dblock%26id%3De2a93533-a982-476f-a829-8a9bae0a34ed%26cache%3Dv2&w=3840&q=75)
근데 CoT는 왜 내려간거지??(→ Reasoning 태스크가 아니니까.)
reasoning 태스크(코드 네임, 로직 그리드 퍼즐) 2개에서는
![notion image](https://inblog.ai/_next/image?url=https%3A%2F%2Fwww.notion.so%2Fimage%2Fhttps%253A%252F%252Fs3-us-west-2.amazonaws.com%252Fsecure.notion-static.com%252F92d052dd-6c78-4861-b0e9-685d1e03b862%252FScreen_Shot_2023-08-16_at_3.09.47_PM.png%3Ftable%3Dblock%26id%3D42604026-d2b4-4671-8387-6db067ce210d%26cache%3Dv2&w=3840&q=75)
![notion image](https://inblog.ai/_next/image?url=https%3A%2F%2Fwww.notion.so%2Fimage%2Fhttps%253A%252F%252Fs3-us-west-2.amazonaws.com%252Fsecure.notion-static.com%252Fa37dfa59-582e-40fa-9677-8228f01bb0e6%252FScreen_Shot_2023-08-16_at_3.10.02_PM.png%3Ftable%3Dblock%26id%3Daa09b6cc-683a-4e47-a305-501871faeda4%26cache%3Dv2&w=3840&q=75)
Analysis
1. SPP는 LLM의 지식, 리즈닝 능력을 올려준다.
2. LLM은 파인튜닝 없이 유용한 페르소나를 정할 수 있다.
Moreover, the fact that SPP-Profile does not outperform SPP in two of the three tasks suggests that a fine-grained name of the persona without a detailed description may already be sufficient for eliciting certain domain knowledge.
3개의 태스크 중에서 2개의 태스크에서 SPP-Profile이 SPP를 이기 못했다는 사실은 특정 도메인 지식을 부여하기 위해서 자세한 설명없이 상세한 이름을 붙여주는 것만으로 충분하다는 사실을 의미한다.
여기서 말하는 fixed-persona는 페르소나를 그냥 Expert라는 광범위한 전문가로 만들어서 사용하는걸 의미한다. 이건 안좋았다고 함.
![notion image](https://inblog.ai/_next/image?url=https%3A%2F%2Fwww.notion.so%2Fimage%2Fhttps%253A%252F%252Fs3-us-west-2.amazonaws.com%252Fsecure.notion-static.com%252F2f4290fd-b4cf-4d01-9f35-33cf7d71ba7c%252FScreen_Shot_2023-08-16_at_3.22.20_PM.png%3Ftable%3Dblock%26id%3D4132cf3c-9aed-4d70-8454-b09303aa63e4%26cache%3Dv2&w=3840&q=75)
실제로 사용한 프롬프트
AutoGPT랑 전혀 다른거구나.. 프롬프트 하나로 그냥 끝내는거네
When faced with a task, begin by identifying the participants who will contribute to solving the task. Then, initiate a multi-round collaboration process until a final solution is reached. The participants will give critical comments and detailed suggestions whenever necessary. Here are some examples: --- Example Task 1: Use numbers and basic arithmetic operations (+ - * /) to obtain 24. You need to use all numbers, and each number can only be used once. Input: 6 12 1 1 Participants: AI Assistant (you); Math Expert Start collaboration! Math Expert: Let's analyze the task in detail. You need to make sure that you meet the requirement, that you need to use exactly the four numbers (6 12 1 1) to construct 24. To reach 24, you can think of the common divisors of 24 such as 4, 6, 8, 3 and try to construct these first. Also you need to think of potential additions that can reach 24, such as 12 + 12. AI Assistant (you): Thanks for the hints! Here's one initial solution: (12 / (1 + 1)) * 6 = 24 Math Expert: Let's check the answer step by step. (1+1) = 2, (12 / 2) = 6, 6 * 6 = 36 which is not 24! The answer is not correct. Can you fix this by considering other combinations? Please do not make similar mistakes. AI Assistant (you): Thanks for pointing out the mistake. Here is a revised solution considering 24 can also be reached by 3 * 8: (6 + 1 + 1) * (12 / 4) = 24. Math Expert: Let's first check if the calculation is correct. (6 + 1 + 1) = 8, 12 / 4 = 3, 8 * 3 = 24. The calculation is correct, but you used 6 1 1 12 4 which is not the same as the input 6 12 1 1. Can you avoid using a number that is not part of the input? AI Assistant (you): You are right, here is a revised solution considering 24 can be reached by 12 + 12 and without using any additional numbers: 6 * (1 - 1) + 12 = 24. Math Expert: Let's check the answer again. 1 - 1 = 0, 6 * 0 = 0, 0 + 12 = 12. I believe you are very close, here is a hint: try to change the "1 - 1" to "1 + 1". AI Assistant (you): Sure, here is the corrected answer: 6 * (1+1) + 12 = 24 Math Expert: Let's verify the solution. 1 + 1 = 2, 6 * 2 = 12, 12 + 12 = 12. You used 1 1 6 12 which is identical to the input 6 12 1 1. Everything looks good! Finish collaboration! Final answer: 6 * (1 + 1) + 12 = 24 --- Example Task 2: Write a poem that meets the following requirements: (1) the poem has seven lines and the first letters of each line forms the word "CHATGPT"; (2) the poem is about explaining what is a quantum computer. (3) the poem needs to be easy to understand by a ten years old kid. Participants: AI Assistant (you); Poet; Computer Scientist; Ten year old child Poet: Make sure that you write the poem with seven lines, and the first letters of the lines should be C, H, A, T, G, P, T. Computer Scientist: A quantum computer is an advanced computing device that uses the principles of quantum mechanics to process and store information. Unlike classical computers that use bits to represent information as 0s and 1s, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously, due to a quantum phenomenon called superposition. You can consider using these information for the poem. Ten year old child: I hope the poem to be fun and easy to understanding. I don't want to see a lot of jargons or complicated concepts. AI Assistant (you): Thanks for the guidance! Here's my initial attempt at the poem: Computational wonder of our age, Harnessing the quantum world's strange ways, Atoms dance, entwined in dual state, Tapping secrets hidden in their haze. Grand power to solve, simulate, Profound problems that perplex the wise, Transforming our future, we await. Poet: Let's verify if the poem meets the requirements. The first letters are CHATGPT which is correct! And the poem rhymes well. Good job! Computer Scientist: Everything looks good to me! Ten year old child: I don't know what does perplex mean. Can you make the use of words easier to understand? AI Assistant (you): Sure, let me revise the poem by using more common words. Check out the revised version: Curious machine of our time, Harnessing the quantum realm's odd ways, Atoms play, two states they embrace, Taking secrets from their puzzling maze. Great power to solve and imitate, Problems that confuse the brightest minds, Transforming our future, we await. Poet: Let's check again if the poem meets the requirements. The first letters are C H A T G P T. And now the poem is more accessible to children. Everything looks good to me. Computer Scientist: Looking good! Ten year old child: I like this version a lot! Finish collaboration! Final answer: Curious machine of our time, Harnessing the quantum realm's odd ways, Atoms play, two states they embrace, Taking secrets from their puzzling maze. Great power to solve and imitate, Problems that confuse the brightest minds, Transforming our future, we await. --- Now, identify the participants and collaboratively solve the following task step by step. Remember to present your final solution with the prefix "Final answer:". Task: Write a short and coherent story about {topic} that incorporates the answers to the following {n} questions: {questions}
실제로 실행해본 결과(너무 대충 실험해본거라 결과 비교 용도는 아니고 과정을 봤습니다)
해당 방법으로 출력한 결과
독자 페르소나를 안만드는데
Participants: AI Assistant (you); Novelist; Creative Writer Start collaboration! Novelist: Let's first decide on the genre of the novel. What kind of story do you want to tell? Creative Writer: We could write a mystery novel with a twist ending, or a drama that explores the complexities of human relationships. AI Assistant (you): A mystery novel with a twist ending sounds exciting! What kind of main character should we have? Novelist: Maybe a detective who is haunted by a personal tragedy, or a mathematician who uses their logic skills to solve crimes. Creative Writer: The protagonist could be a journalist who stumbles upon a conspiracy while investigating a seemingly unrelated story. What do you think? AI Assistant (you): I like the idea of a journalist as the protagonist, it gives us a unique perspective to explore the mystery. Novelist: Now let's start writing! Here's my initial attempt at the opening paragraph: The crisp autumn air bit into Jane's cheeks as she made her way down the dark alley. Her heels clicked against the cobblestones as she approached the lone figure shrouded in shadows. "What's going on here?" she asked, the click of her pen echoing through the stillness. Creative Writer: I like it, but what if we start the novel with something more gripping and attention-grabbing? Like a high-stakes moment that sets the tone for the rest of the book. AI Assistant (you): That's a great idea! Here's my revised opening: The car hurtled down the dark, deserted road, its headlights illuminating the thick fog that obscured everything beyond ten feet. Maria gripped the steering wheel with white knuckles, trying to calm her racing heart. She knew she was being followed, but who? And why? She glanced at the rearview mirror, her breath catching in her throat at the sight of the black sedan bearing down on her. Novelist: I like where this is going! Let me add the next part: She gunned the engine, the neon-lit sign of a motel just ahead. With skillful precision, she slid into the parking lot and yanked out her phone. Her fingers fumbled for the right number as she darted inside the dimly lit lobby. "I need your help," she hissed, her eyes darting toward the door. Creative Writer: Let's make it even more intriguing! What if the person on the phone is not who they seem to be? AI Assistant (you): Great idea! Here's my revised version: She gunned the engine, the neon-lit sign of a motel just ahead. With skillful precision, she slid into the parking lot and yanked out her phone. Her fingers fumbled for the right number as she darted inside the dimly lit lobby. "I need your help," she hissed, her eyes darting toward the door. A voice on the other end of the line spoke in a language she didn't recognize, sending a chill down her spine. She turned around, but it was too late - the door burst open, and a group of men in black suits swarmed inside. Novelist: Love it! Let me add the next part: Maria's heart pounded in her chest as she tried to back away, but strong hands grabbed her from behind. She tried to scream, but a cloth covered her mouth, and she inhaled a sickly sweet odor. The room started to spin, and she felt herself falling into darkness. Finish collaboration! Final answer: She gunned the engine, the neon-lit sign of a motel just ahead. With skillful precision, she slid into the parking lot and yanked out her phone. Her fingers fumbled for the right number as she darted inside the dimly lit lobby. "I need your help," she hissed, her eyes darting toward the door. A voice on the other end of the line spoke in a language she didn't recognize, sending a chill down her spine. She turned around, but it was too late - the door burst open, and a group of men in black suits swarmed inside. Maria's heart pounded in her chest as she tried to back away, but strong hands grabbed her from behind. She tried to scream, but a cloth covered her mouth, and she inhaled a sickly sweet odor. The room started to spin, and she felt herself falling into darkness.
그냥 GPT 3.5로
As the sun began to set over the small town of Millfield, Indiana, Rachel Porter eagerly made her way through the familiar streets towards her childhood home. Though she had lived in the city for the last decade, Rachel still found herself returning to the quiet comfort of her hometown whenever she could. This time, however, Rachel had more than just the familiar sights and sounds of her childhood to look forward to. She was returning home for her ten-year high school reunion, an event that had her feeling both excited and a little anxious. With each step, Rachel felt the weight of the years fall away, leaving her feeling like the nervous teenager she remembered. She couldn't help but wonder what had happened to her old classmates, and whether they too were feeling the same mixture of nostalgia and apprehension. As she finally reached the gate of her childhood home, Rachel took a deep breath and pushed open the familiar wooden door. The moment she stepped inside, she was enveloped by the scent of freshly baked cookies and the sound of her mother's joyful laughter. "Rachel, darling, you made it!" Her mother cried, wrapping her in a tight embrace that felt like coming home. "I've got so much to tell you and so much to catch up on. But first, let's get you settled in." With a heart full of love and excitement, Rachel followed her mother down the hallway towards her childhood bedroom. As she stepped inside, she felt a wave of memories wash over her. From the familiar posters on the wall, to the well-worn textbooks scattered on the desk, everything looked just as she had left it. As she sat down on the bed and took it all in, Rachel couldn't help but feel a sense of excitement for the reunion that lay ahead. She knew that the coming days would be filled with laughter, tears, and memories that would last a lifetime. The only question now was: where would the journey take her?
Share article
Subscribe to our newsletter