論文紹介：Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation

Daichi Horita, Naoto Inoue, Kotaro Kikuchi, Kota Yamaguchi, Kiyoharu Aizawa, "Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation" arXiv2024(CVPR2024)

https://arxiv.org/abs/2311.13602

1. Retrieval-Augmented Layout
Transformer
for Content-Aware Layout Generation
Daichi Horita, Naoto Inoue, Kotaro Kikuchi, Kota Yamaguchi,
Kiyoharu Aizawa, CVPR2024
pՖ?A?6??.?A6?f?Qn??2024/05/30

2. ?O?
n]9i?HW
•3Mfm沩m???Goi?
•l?V
•?O?i?)|?TdA*?۔'課??
BB??|?i?HW
ngoeo
•-?% ?<"}?4Ti?HWS8\?'c<
•% B1<"}???ns??i???i?HW
n??-
•ZB}??E?N??lmf?b[?
•<f<???<"}??f??i?HWS8\?Uvu??%-?+?
→T>x?I?CV4Ti?HW(RAG)?????C?,Q???
]9i?HW?'F[Donovan+, CHI2015]
Retrieval-Augmented Generation

3. T>x?I?CV4Ti?HW?3FUSJFWBM"VHNFOUFE(FOFSBUJPO?
nRAG??
•6???V????S8\(LLM)???7*9i?HW??6ٟ'E?5T?T>x?x??.??
?????3Qt?w?B1.?%???I?o
•,?Q?
•LLM?+s,qP?N??E?5T?P?N?
•h?P??m?????(?gT?i?FW?4\2moXd??T?
nR[49
•=x??>49`m ڰ?D??ף?
•DR???Dt??6ٟ'RS[T>x?
RAG

4. ???}n?r?
n$d7d5'c<???]9
i?HW
•e<??*?,qTdA*.????i?HW
[Yang+, ACM TOMM2016]
•NNJm*i?HW[Li+, ICLR2019] [Arroyo+,
CVPR2021]
•??A??H^m2
•GAN
•VAE
•Diffusion Model
n$d7d5~G!??]9
i?HW
•ContentGAN[Zheng+, ACM TOG2019]
•i?)|,Od7 *.f?*???
•CGLGAN [Zhou+, IJCAI2022]
•???A3QAo8$m1ެ;?7D?O4H
=u*?

5. l?l3
n]9i?HW????T>x?I?CVi?HW(RAG)
•T>x??hCZ???8m0%? 0??-?=Z+Z
nT>x?I?CV4Ti?HW'3i???H?Z?(RALF)?K R?

6. K R?H?Z?

7. K R?H?Z?

8. K R?H?Z?

9. K R?H?Z?

10. K R?H?Z?

11. Image encoder
CGL-GAN[Zhou+, CVPR2022]

12. Retrieval augmentation module

13. Constraint encoder

14. Layout decoder
Transformer
decoder

15. <??f?<?
n8m0,49
•ENw?i?沏?3bi??3Mfm
•CGL [Zhou+, IJCAI2022]
•?7m(XdP???60,548
•?7m(Xdb[??1,000
•??x?N??100
•PKU [Hsu+, CVPR2023]
•?7m(XdP???9,974
•?7m(Xdb[??905
•??x?N??100
n?P'kI?V
•!ZF4 R9[ *
•FID↓
•d1m]P?,?D?(Und↓)
•mAm]P?,?D?(Ove↓)
•$d7d5R9[ *
• \m)Xd?Occ↓?
•?????*$?Rea↓?
n??x??<?
•K??(???K=16
•*?,qi?)|
•???350, A?240

16. <??fx?Q?
n<??l3YB?3

17. <??fx?Q?
n<?D?l3YB?3

18. T>x?I?CV?P?b[???D?YB?3

19. T>x?x?Q???????<?D?l3YB?3

20. ???
n]9i?HW????T>x?I?CVi?HW(RAG)
•T>x??hCZ???8m0%? 0??-?=Z+Z
nT>x?x?Q???????YB?3
•K?6???N??F](G\?i?HW?.??
n??-
•??NT>x?x?Q?y4.??d&dG\H^m2?
•/???.?%?.??D?
•T>x?S1[7?6?U?-+
•Dti?u?3qA???
•??????

21. ?h? ??Nb

22. <??fx?Q??Wm'm+?x\???
C : 7%[S : &+P : N)(Xd

23. <??fx?Q??Wm'm+?x\???

24. <??fx?Q??Wm'm+?x\???