Bold Colors, Clear Subjects_ A Deep Review of USO, the Unified Customization Model.pdf

USO arrives with a simple promise that cuts through confusion in image generation. It unifies two worlds that are usually handled apart: style-driven image creation and subject-driven preservation. Most tools force a choice between matching a look or keeping a subject consistent. USO treats both as parts of the same equation. It separates content from style and lets you remix them with intent. The result is a practical system for stylization, identity preservation, layout control, and mixed multi-style rendering without asking you to fight the model.

1. Bold Colors, Clear Subjects: A Deep Review of USO,
the Unified Customization Model

Why USO Matters Right Now
USO arrives with a simple promise that cuts through confusion in image generation. It unifies two
worlds that are usually handled apart: style-driven image creation and subject-driven
preservation. Most tools force a choice between matching a look or keeping a subject consistent.
USO treats both as parts of the same equation. It separates content from style and lets you remix
them with intent. The result is a practical system for stylization, identity preservation, layout
control, and mixed multi-style rendering without asking you to fight the model. USO is open-sourced with code, inference scripts, model weights, and a demo app. It is built to
run on consumer GPUs with an fp8 mode and an offloading path that brings VRAM down to a
workable range. It has a triplet training setup, a disentangled learning scheme, and style reward
learning to avoid the blunt tradeoffs seen in ordinary adapters. It is a project with a clear point of
view and careful mechanics behind it.
Access the repository and demo here: https://github.com/bytedance/USO

What USO Is, In Practice
The Core Idea
USO treats style as a learnable signal that can be aligned and rewarded, while keeping content
separate and protected. It does this through:

2. ●​A triplet dataset: content images, style images, and stylized results that link the two.
●​A disentangled learning scheme that keeps content features and style features apart.
●​A style reward-learning component to shape outcomes toward human-perceived style
quality.
That combination solves real pain: you can keep a person’s face reliable and still apply strong
stylization. You can copy layout without flattening textures. You can blend multiple styles without
washing out identity. USO is built for repeatable customization rather than one-off prompt luck.
What You Can Do With It
●​Subject-driven generation: keep a subject consistent and place it in new scenes.
●​Style-driven generation: match a specific look with minimal prompt contortions.
●​Style plus subject: steer both at once for controlled remixes.
●​Layout-preserved generation: leave prompt empty and use references to preserve
composition.
●​Multi-style synthesis: blend more than one style reference in a single pass.
●​High-detail portrait rendering: preserve skin detail while moving across lighting and
framing setups.
USO supports these use cases with simple CLI parameters and a Gradio app.

Setup and The Road to First Image
Requirements and Installation
●​Python between 3.10 and 3.12.
●​Torch 2.4.0 with the specified torchvision build for CUDA 12.4.
●​A virtual environment is encouraged.
Command sketch:
●​Create venv and activate.
●​Install torch from the given index URL.
●​Install the project requirements with pip.
●​Create a .env file from example.env and add a Hugging Face token.
●​Download weights with the provided downloader.
USO includes an fp8 path with offloading that targets 16 to 18 GB peak VRAM depending on
reference count. That makes it realistic to run on a single modern consumer card.
The Inference Script
The entry point is inference.py. You pass a prompt, a list of image paths, width and height, and
optional flags like offload and model type. The first image path is treated as the content
reference. If you want style-only and no content identity, you leave the first path empty and pass
style references after that.

3. Common patterns:
1.​Subject-driven:​

○​One content reference image.
○​Prompt describes an activity or scene for the subject.
2.​Style-driven:​

○​No content image.
○​One or more style images and a concept prompt.
3.​Style plus subject:​

○​One content image.
○​One or two style images.
○​Prompt for scene and context, or leave empty to preserve layout.
4.​Multi-style:​

○​No content image.
○​Two style images to blend.
○​Short prompt, then adjust resolution as needed.
5.​Low VRAM:​

○​Add --offload and set model_type to flux-dev-fp8.
○​Expect a small performance cost in exchange for lower memory footprint.
The script’s parameterization is minimal. The model is designed to infer structure and style
alignment from references rather than from prompt stunts.
The Gradio App
There is a simple app.py script. You can run it straight from the repo. It supports the fp8 mode
and offloading flags just like the CLI. You set the environment variable for the fp8 path and
launch. Peak VRAM sits near 16 GB with a single reference, slightly higher with multi-style.

The Training Framing and Why It Works
The Triplet Dataset
USO constructs a dataset of three pieces per training example:
●​A content image.
●​A style image.
●​The corresponding stylized content image tying the two together.
This unlocks a meaningful supervision signal. The model sees not just a style image and a
guess, but the expected result that merges the content’s structure with the style’s appearance.
That makes alignment during training less fuzzy.

4. Disentangled Learning
The core of USO’s approach is to keep content and style representations separate. Many
systems let these signals bleed into each other. This leads to overly rigid outputs that bake in
concrete details of the style example or, on the other side, outputs that drift off and lose the
intended look.
USO promotes two complementary objectives:
●​Style alignment: ensure that extracted style features align with the stylized result.
●​Content–style separation: reduce leakage so that content does not deform into style and
vice versa.
That translates into better control during inference. You can dial in style references without
unexpectedly changing face structure or pose. You can keep layout intact while still shifting local
texture and color. Portrait retention benefits strongly from this separation.
Reward Learning for Style
USO adds a style reward-learning component. This gives the model a scalar sense of “how
much” or “how well” a generated image reflects the intended style. It tunes decisions during
training and reduces brittle behavior like oversmoothing or overfitting to a single stylistic element.
Reward-guided training is tricky to get right. If the reward is naive, you get over-optimized outputs
that feel fake or repetitive. USO’s reported results show that the style scoring aligns with what
users want from stylization: consistent palettes, characteristic brushwork or grain, and style
structure that does not override identity.

Real-World Behavior and Image Quality
Subject Fidelity
Subject-driven generation is the standout. USO keeps face geometry and key identity cues while
moving the subject into new settings. Best practice:
●​Use a half-body input for half-body prompts.
●​Use a full-body input when pose or framing changes a lot.
●​Keep prompts simple and concrete about action and environment.
Portraits show strong skin detail, which many systems wash out when style is applied. USO
avoids that common flattening effect by treating style as a layer rather than a rewrite.
Style Strength and Nuance
Style references can be light-touch or heavy-handed depending on your prompt and the
references you pick. A single style image yields a coherent look. Two style images deepen the

5. palette and texture blend but can clash if they fight each other. USO manages blends without
collapsing into mush. You still need to pick complementary references.
The key observation: the model replicates style structure without copying literal elements. It
catches color grading, brush shape, line weight, and rendering density while preserving the
scene you describe.
Layout Control
If you want layout preservation, use the content reference and leave the prompt empty. USO
keeps composition, spatial arrangement, and perspective while applying style. This is rare to see
handled cleanly, since many models require heavy prompt engineering or mask work. USO
builds layout preservation into the default behavior when the prompt gives no competing
instruction.
Multi-Style Composition
Multi-style is where the model’s disentanglement shines. It does not average styles in a naive
way. Instead, it picks up commonality where styles overlap and retains characteristic features
where they diverge. Color harmony is the biggest driver of success here. If two styles have
opposing grading, you may need prompt nudges about lighting conditions to anchor the blend.

Performance and VRAM
fp8 Mode and Offloading
USO supports an fp8 model type with offloading to reduce VRAM to around 16 to 18 GB during
inference. That covers single-reference and multi-reference runs. This option is a practical gift for
single-GPU users.
Tradeoffs:
●​Slight runtime increase from offloading.
●​Small quality variance in edge cases with extremely detailed textures.
The cost is reasonable given the access it provides. You can run the full pipeline without needing
data center hardware.
Resolution and Aspect Ratios
The examples target 1024 by 1024. You can push higher, but VRAM and time costs grow. When
changing aspect ratio, pick references with similar framing. A portrait reference stretched into a
panoramic landscape tends to bend identity cues. Keep aspect ratios consistent with your
content reference for the cleanest results.

6. The CLI Patterns That Work
Subject-Driven
●​One content reference as the first image path.
●​A clear prompt with the action and environment.
●​Avoid poetic phrasing that confuses the model. Use direct nouns and verbs.
Example structure:
●​Prompt: The person is reading by a window with soft afternoon light.
●​Image paths: [content.jpg]
●​Flags: --width 1024 --height 1024
Style-Driven
●​First image path empty.
●​One or two style images after the empty slot.
●​A concise prompt with subject and scene.
Example structure:
●​Prompt: A small cafe on a rainy street corner.
●​Image paths: [empty, style1.webp]
●​Flags: --width 1024 --height 1024
Style Plus Subject
●​First reference is the content image.
●​One or two style images next.
●​Prompt optional for layout control.
Example structure:
●​Prompt: The woman addresses a crowd from a podium.
●​Image paths: [identity.webp, style.webp]
●​Flags: --width 1024 --height 1024
Multi-Style
●​First path empty.
●​Two style images after the empty slot.
●​Keep the prompt short to let styles take the lead.
Example structure:
●​Prompt: A mountain village at dawn.
●​Image paths: [empty, styleA.webp, styleB.webp]
●​Flags: --width 1024 --height 1024

7. Low VRAM Path
●​Add --offload and set model_type to flux-dev-fp8 for inference.
●​For the Gradio app, use --offload and --name flux-dev-fp8 and set FLUX_DEV_FP8 path.

Practical Tips for Better Results
Choosing Content References
●​Match the framing to your desired output. Half-body for half-body prompts.
●​Use sharp, well-lit images for identity. Make the model’s job easier.
●​Avoid occlusions that hide distinct facial features when identity is central.
Picking Style References
●​Choose styles with clear palettes and texture language.
●​Avoid conflicting lighting directions when blending multiple styles.
●​Use two style references for richer looks. Keep them consistent in era, medium, or
technique.
Prompt Writing
●​Keep prompts declarative and specific.
●​State subject, action, setting, and light if relevant.
●​Skip filler and metaphor. The model responds best to concrete nouns and clean verbs.
Layout Preservation
●​Leave the prompt blank to keep layout from the content reference.
●​If you need minor changes, keep the prompt minimal, like “nighttime” or “snowfall.”
Troubleshooting
●​If identity drifts, raise image resolution or pick a clearer content reference.
●​If style is too faint, add a second style reference or describe color grading in the prompt.
●​If outputs look busy, shorten the prompt to reduce conflicting cues.

What Sets USO Apart Technically
Clean Separation of Content and Style
The disentanglement is not a side note. It is a training objective embedded into the pipeline. This
stabilizes identity and composition across style changes without needing heavy prompt
management or extra controls.

8. Reward Shaping That Reflects Style
Style reward-learning acts like a teacher that nudges the model toward human judgments of style
strength and coherence. That reduces two frequent errors:
●​Overly literal style copying that contaminates content geometry.
●​Overly soft style application that reads as generic color shifts.
Triplet Data That Guides Composition
By training on content, style, and the expected stylized result, USO does not rely solely on latent
alignment tricks. It sees grounded examples where structure must remain while appearance
shifts. This pairs well with reward learning and gives the network a clear map for what to keep
and what to change.

A Walkthrough: From Zero to First Batch
1.​Create and activate a virtual environment with Python 3.10.
2.​Install PyTorch 2.4.0 and torchvision 0.19.0 from the CUDA 12.4 wheel index.
3.​Install requirements with pip -r requirements.txt.
4.​Copy example.env to .env and add a valid Hugging Face token.
5.​Install huggingface_hub and run the weights/downloader.py script.
6.​Prepare references in assets or your own folder. Keep naming tidy.
7.​For subject-driven generation, run inference.py with a content reference and a clear
prompt.
8.​For style-driven generation, leave the first image path empty and pass style images.
9.​For style plus subject, combine both references and adjust prompt length.
10.​If VRAM is tight, add --offload and use the fp8 model type.
Expected time to first image on a single consumer GPU: a handful of minutes after setup,
depending on downloads and disk speeds.

Where USO Fits in Creative Workflows
Portrait and Lifestyle Work
USO is well suited to portrait-driven projects where the person must remain recognizable across
scenes. You can portray the same subject in varied lighting, backgrounds, and visual treatments.
Skin detail stays intact, and the face geometry holds up under stronger stylizations.
Editorial and Concept Art

9. Use style-driven output to generate consistent lookbooks. Feed a few style images and prompt
scene ideas. Because USO preserves layout well when desired, you can keep the composition
from a sketch or a mood board and convert it into a finished style layer.
Product and Branding Exploration
For product mockups, treat the product photo as the content reference. Apply one or two visual
styles to frame the product in different contexts. Layout-preserved runs keep proportions stable
while changing material feel and color grading.
Previsualization
When speed matters, the fp8 path lets teams run many variations. Build a grid with different style
blends and pick the best. Keep the prompt short so references dominate. Save successful pairs
for future reuse to keep visual language consistent across a project.

Model Behavior Under Stress
Extreme Style References
If you feed wildly abstract or heavily patterned styles, USO keeps subject integrity better than
typical adapters. You might still see edge cases where the style tries to imprint shapes on facial
features. Mitigation:
●​Slightly lengthen the prompt with a plain line about clear facial details.
●​Pick a content reference with more frontal clarity and consistent lighting.
Large Pose Shifts
If the content reference is a close-up and you prompt a distant full-body scene, identity will
degrade. USO helps, but representation gaps are real. Use a full-body content image when
planning to change framing drastically.
Conflicting Multiple Styles
Two references with opposite palette rules or brush semantics can pull the model in different
directions. If that happens:
●​Narrow the prompt to a clear lighting condition.
●​Switch one style reference to a variant with neutral grading but similar texture traits.

Licensing, Ethics, and Use Considerations

10. USO is released under the Apache 2.0 license. Respect licenses for any base models you might
swap in. The team emphasizes responsible use and compliance with local laws. The dataset for
training leans on generated and open datasets. If issues arise around specific content, they invite
contact for removal.
This project is made available for research and creative workflows. It is capable of detailed
identity preservation. Use it with consent and care, especially for real people. Keep records of
references used for auditability within teams or clients.

Clarity for Teams: Repeatable Setup
Environment Pins
●​Pin Python to 3.10 for uniformity across machines.
●​Pin torch and torchvision versions exactly as recommended.
●​Keep a shared .env template and a process for distributing tokens securely.
Weight Management
●​Use the downloader script and version control for the weights folder structure.
●​For shared workstations, set a cache path and document it.
Naming and Asset Folders
●​Build a reference library with consistent naming, like subject_name_shotType.jpg and
style_label_medium.webp.
●​Keep a readme in your assets directory listing the pairings that work well.
Reproducible Commands
●​Store CLI commands in small shell scripts per project.
●​Include width and height, model_type flags, and offload settings in those scripts.

Subtle Tricks That Improve Output
●​If the style overwhelms small details, add a second style image with a gentler texture
treatment that still shares palette cues.
●​If the subject’s eyes lose clarity, prompt a short phrase like “sharp eyes” and avoid extra
descriptors.
●​For fabric realism, prompt a fabric type and weave density in a single line, and let the
style reference handle the rest.
●​To maintain composition in complex scenes, run a layout-preserved pass first with an
empty prompt, then a second pass with a brief prompt to insert environmental cues.

11. What Stands Out in Day-to-Day Use
●​The prompt can be short. The references do heavy lifting. That saves time.
●​Identity lock is the strongest part. Face shape and expression remain recognizable.
●​The style reward learning avoids dead-flat grading. You get a more cohesive look.
●​fp8 mode with offloading makes single-GPU use realistic without awkward
micro-optimizations.
●​Multi-style blending is reliable when references have shared color logic.

Common Missteps and Fixes
●​Overwritten Identity​

○​Cause: weak or occluded content reference.
○​Fix: sharper reference, front-facing, matching framing.
●​Muddy Style​

○​Cause: vague prompt and weak style signals.
○​Fix: add a second style reference or state dominant palette.
●​Composition Drift​

○​Cause: long prompt fighting layout with new objects and strong verbs.
○​Fix: leave prompt empty for a first pass to lock layout.
●​Over-detailed Noise​

○​Cause: two style images with competing micro-textures.
○​Fix: replace one style with a simpler reference that keeps palette but reduces
texture density.

The Repo, Structure, and How to Navigate
●​README.md gives the overview, setup steps, usage patterns, and memory notes.
●​inference.py handles the generation routine and flags.
●​app.py launches the Gradio interface with optional offload and fp8 name.
●​weights/downloader.py automates model checkpoint retrieval after you set your Hugging
Face token in .env.
●​assets folder contains working example references and output comparisons for sanity
checks.
●​requirements.txt enumerates dependencies suited to the recommended torch build.
For most users, the path is:

12. ●​Clone repo.
●​Set up environment.
●​Download weights.
●​Run app.py or inference.py with the patterns above.
Access the repository and run your first tests here: https://github.com/bytedance/USO

Benchmark Mindset Without the Buzzwords
USO’s value is plain in side-by-sides with simple tasks:
●​Take a crisp headshot as content. Target a strong ink style. Check skin detail and facial
landmarks. USO holds shape and specular highlights.
●​Feed an architectural exterior as content and a watercolor style. Look at windows and
rooflines. USO keeps straight edges while applying loose pigment patterns.
●​Use two styles with a shared beige-blue palette. Look for color harmony and brush
cohesion. USO merges without gray mush.
●​Blank prompt with content reference. Inspect composition fidelity. USO retains pose and
negative space as expected.
You do not need a lab to see the gains. The differences are visible in the first batch.

Integration Thoughts for Teams
●​Command Wrappers​

○​Wrap common runs in shell or Python scripts.
○​Include date and style tags in output filenames for traceability.
●​Asset Tracking​

○​Catalog style references and the scenarios where they perform well.
○​Keep a small gallery of “golden pairs” for quick checks after updates.
●​Model Updates​

○​Record torch and driver versions.
○​Re-run a standard preset suite after any dependency or weight change.
●​Collaboration​

○​Use the Gradio app for quick internal reviews.
○​Move to CLI scripts for batch jobs and automation.

Limitations and Sensible Expectations

13. ●​Extreme changes in framing break identity. Match reference to goal framing.
●​Conflicting multi-style pairs create noise. Pick styles that share palette logic or subject
matter cohesion.
●​fp8 and offloading introduce minor slowdowns. Plan batch runs accordingly.
●​Out-of-distribution style forms may produce partial adherence. Improve with a second
style reference or prompt color anchors.
These are practical constraints. They are manageable with a small set of habits.

A Short Guide to Ethical Use
●​Get consent for real-person identity use.
●​Avoid misrepresentation. Label stylized or composited renders in contexts where it
matters.
●​Keep reference provenance. Track who owns what.
●​Respect model and dataset licenses. Apache 2.0 covers this repo, but honor any external
base model terms.

What You Will Appreciate After a Week of Use
●​Predictable subject handling reduces manual cleanup.
●​Fewer failed runs from messy prompts. You can write short and get results.
●​Style reward behavior gives color and texture a coherent feel without endless tweaking.
●​Layout-preserved generation is a reliable tool for design iteration.
●​fp8 lowers the bar to entry for local experimentation.
The learning curve is gentle. The outputs carry a stable logic once you align references and
framing.

Getting Started Checklist
●​Install Python 3.10, torch 2.4.0, and project requirements.
●​Create .env and insert a valid Hugging Face token.
●​Download weights via the provided script.
●​Collect your content and style references in a clean folder structure.
●​Test four core scenarios:
1.​Subject-driven with a half-body portrait.
2.​Style-driven with one strong style reference.
3.​Style plus subject with one content image and one style.
4.​Layout-preserved with empty prompt.
●​Test multi-style with two related style references and a short prompt.
●​If VRAM is tight, run with --offload and model_type flux-dev-fp8.

14. The Bottom Line for Practical Adoption
USO is an open project that treats style and subject as cooperative signals. It trains with triplets,
enforces disentanglement, and uses reward learning to shape style behavior. The toolset is
approachable: a CLI that behaves, a Gradio app for quick trials, and a memory profile that works
on a single GPU when using fp8 and offloading.
For teams and solo creators, the path from setup to useful output is short. For portraits and
layout-sensitive work, the results stand out. For style exploration and blends, the references do
the heavy lifting, letting you write simpler prompts and get consistent images.
Access the project, read the README, and run the examples here:
https://github.com/bytedance/USO

Expanded Notes for Power Users
Dataset Design Mindset
Triplet data ties expected results to specific content and style pairs. This carries two benefits:
●​It teaches the model what to change and what to preserve.
●​It calibrates reward learning because the target stylized image is not a random guess.
Even small improvements in triplet quality improve alignment. If training is released end-to-end,
that part of the pipeline will matter most to custom builds.
Disentanglement Objectives
Strong disentanglement needs explicit losses that penalize leakage. In practice, you want:
●​A feature space where content features predict structure and identity.
●​A separate space where style features predict palette, stroke, and material cues.
●​Cross-terms that discourage style from altering structure and content from swallowing
palette.
Conversions at inference time must compose these spaces without flattening. That is what drives
USO’s steady identity retention.
Reward Learning Details
Reward learning hinges on a reliable score. Options include learned discriminators or proxy
measures trained on human preference data. The key is to avoid optimizing to the score in a way
that removes diversity. USO’s visual outputs suggest the reward function balances strength with
natural variation. That balance is rare and valuable.

15. VRAM and Throughput Planning
●​Expect 1024 by 1024 single-reference generation to sit in the 16 to 18 GB range under
fp8 with offload.
●​Batch size is constrained. Run sequential jobs with a queue to keep machines stable.
●​If you need higher resolution, consider tiled approaches only after testing native runs.
Tiling can complicate style continuity.

Editorial Verdict
USO delivers on a simple premise. Treat content and style as separate levers and train the
model to respect both. The results back that up. Identity stays intact. Style reads convincingly.
Layout preservation works with minimal instruction. The CLI is straightforward. The Gradio app
makes quick looks easy. The fp8 mode lowers the hardware barrier for individuals and small
teams.
If you rely on style consistency or subject fidelity, USO belongs in your toolkit. If you are exploring
multi-style blends for a signature look, USO gives you a stable foundation. The training design
makes sense, and the outputs reflect it.
Get the code, run the demo, and test with your own references:
https://github.com/bytedance/USO