Guiding the Latent Space of Generative Models on Subjective Human Factors.pptx

Generative Artificial intelligence (AI) specifically Deep Generative Models (DGMs) has shown significant potential for generating engineering product designs. While prior research demonstrates DGMs’ ability to incorporate performance objectives, many creative design challenges lack well-defined objective functions. This paper explores the relationship between key physical parameters of bicycles and their usability, aiming to guide a DGM in generating or enhancing designs based on usability. A crowdsourced dataset of bicycle ratings is gathered and used to train a Support Vector Machine (SVM), achieving 95\% test accuracy, to identify critical parameter relationships. The SVM identifies three key parameters that effectively determine a bicycle's usability, namely: Saddle Height, Stack, and CS Textfield. Additionally, Kernel Density Estimation (KDE) with Gaussian kernels is applied in the latent space of a Variational Autoencoder (VAE) to model the usability labels. These KDEs provide continuous log-likelihood scores across the latent space, enabling the classification of each latent point. While latent classification achieves 75\% test accuracy, usability enhancements to the bicycle are effective, as validated by the high-accuracy SVM model. This method of implicitly capturing parameter relationships with usability mitigates the need for exhaustive testing when attempting to uncover them explicitly. Furthermore, KDE guidance in the latent space offers designers precise control over the extent of changes made to an original design, while guided interpolation provides the flexibility to introduce a second design to act as inspiration, enabling the exploration of the design space between them.

1. Guiding the Latent Space of Generative Models on Subjective Human Factors By: Fabien Chiotti, Utrecht University, 2025 Supervisor: Dr. I. Lykourentzou

2. Introduction Motivation Deep Generative Models (DGMs) can incorporate clear, measurable objectives or conditions. Plane Images: Source Bike Images: Source Usable? Aesthetic? The aim is to create usable bicycle designs exportable to BikeCAD for designers. Difficulty lies in defining and quantifying subjective goals. Human-AI collaboration bridges this gap by guiding AI to account for subjective factors like usability.

3. Introduction Why Usability? Usability, once seen as a bonus , is now an expectation among customers. Manufacturers emphasize "people-oriented" and "ergonomic designs" for products involving body contact or manual operation . Design firms heavily invest in design and redesign , e.g. automotive companies spend $1B+ annually on average. A DGM understanding subjective qualities can: Speed up design iterations. Enable personalized design options. Align better with consumer preferences. Enhance efficiency and product appeal.

4. Background The DeCoDE lab has worked on incorporating additional objectives like feasibility and objective functions into DGMs. As part of this research, the BIKED dataset was released. These data points are derived from numerous CAD files created with BikeCAD . The dataset includes 4,512 bicycle models, each represented by BikeCAD data, bike-style labels, and individual component pictures. Bike Morph: Source

5. Background Previous Work Past work of our lab explored key bicycle parameters influencing subjective qualities. Utilized machine learning techniques Regression for quality level assessment. Classification for quality possession. Challenges Limited dataset of 43 bicycles hindered effective feature identification. Low inter-annotator agreement on user ratings. Unclear interactions among identified parameters. Explored 6 subjective qualities with uneven and limited ratings .

6. Research Questions If a bicycle's subjective quality criteria are known, new designs can be generated to meet them. RQ1: From a set of usable/unusable bicycle designs, can parameter relationships, i.e. saddle length and handlebar height, be identified that determine its usability label? Explicitly uncovering parameter interactions is time-consuming. Restricted to statistical analysis or explainable machine learning models. Instead of explicitly modelling parameter conditions for usability: RQ2: Given a set of usable/unusable bicycles, can a generative model be guided to create new designs or improve existing ones to reflect the desired quality?

7. Overview Three main parts: Collecting a set of usable and unusable bicycles. Uncovering parameter relationship with usability. Developing a method for guiding DGMs in generating or enhancing bicycle designs.

8. Collecting a Set of Usable and Unusable Bicycles

9. Collecting a Set of Bicycle Designs for User Ratings 200 bikes sampled from BIKED. Focusing on 71 visually important parameters. The parameter value ranges of these 200 bikes cover 78% of all bikes in BIKED. Sampling aims for an even distribution across all parameters. Sampled vs. BIKED distribution with maximum and minimum values. Distribution of parameter values for sampled bikes.

10. Identifying Usable/Unusable Bicycles Crowdsourcing User Ratings Prolific platform to gather participants. Web-app originally developed by Alice Vitali with some adaptations. Gamification: Tinder-like swipe interaction makes process intuitive and enjoyable, encouraging sustained user participation. Swipe ‘Yes’ or ‘No’ based the bicycle’s usability. BIKES2RATE: Source

11. Identifying Usable/Unusable Bicycles Country List Participants with bicycle experience. Ipsos report “ Cycling Across the World: A 28-country Global Advisor survey” Provides insights into cycling trends, attitudes, and behaviors across various 28 countries. BIKES2RATE: Source Cycling Across the World: A 28-country Global Advisor survey : Source

12. Rating Summary and Results Collection Summary 200 bikes divided into 4 groups of 50. In each group, 60 Prolific participants rated all 50 bikes. Valid Raters Minimum rating time: 90 seconds. Average raters per group: 50.75. Interrater Agreement Average Fleiss Kappa across all groups increased to 0.230 from Vitali’s 0.024 .

13. Examples of Identified Bicycles Usable Unusable

14. Overview Three main parts: Collecting a set of usable and unusable bicycles. Uncovering parameter relationship with usability. Developing a method for guiding DGMs in generating or enhancing bicycle designs.

15. Key Parameters Affecting Usability Data split into 80% training and 20% testing. Applied Support Vector Machines (SVM). Achieved 90% cross-validation accuracy on the training set with key parameters: ’CS textfield ’, ’Saddle height’, and ’Stack’. Achieved 95% test accuracy .

16. Key Parameters Affecting Usability Saddle Height: Vertical distance from bottom bracket to saddle’s deepest point; critical for comfort and pedalling efficiency . Stack: Vertical distance from bottom bracket to head tube top; defines rider posture and balance between comfort and aerodynamics . CS Textfield (Chain Stay Length): Distance from bottom bracket to rear axle; impacts stability, and handling.

17. Research Question 1 RQ1: From a set of usable/unusable bicycle designs, can parameter relationships, i.e. saddle length and handlebar height, be identified that determine its usability label? YES!

18. Overview Three main parts: Collecting a set of usable and unusable bicycles. Uncovering parameter relationship with usability. Developing a method for guiding DGMs in generating or enhancing bicycle designs.

19. Guiding Latent Space Towards Usability Parametric VAE Encoder: Maps parameter space to latent representation. Decoder: Reconstructs parameters from latent representation.

20. Guiding Latent Space Towards Usability Why use Latent Space? Encodes the full 2395 -parameter space, beyond the reduced 71 -parameter subset. Structured, regularized representation of data: Groups similar samples while preserving meaningful variations. Enables meaningful adjustments that can impact multiple physical parameters simultaneously. This is beneficial when similar designs are prone to exhibit the same qualities. Implicitly capture parameter-usability relationships.

21. Guiding Latent Space Towards Usability Proposed Approach Use Kernel Density Estimation (KDE) to capture usable/unusable bike regions in latent space. Navigate latent space guided by these regions. Figure: Sampling from captured regions Figure: Capturing important regions

22. Guiding Latent Space Towards Usability KDE for Classification Two KDEs: Separate KDEs fitted on usable and unusable bikes. Each KDE provides continuous log-likelihood scores for all latent points: Classification rule: A point is classified as usable if: otherwise, it is classified as unusable. Achieved 75% test accuracy on 20 samples with just 61 training samples in a 128-dimensional space.

23. Guiding Latent Space Towards Usability Navigating Latent Space Compute normalized gradients for each KDE at a given point: Aim: Return a usable bike with minimal changes to the original.

24. Guiding Latent Space Towards Usability Best Direction Balance the trade-off by combining gradients to: Increase usable-KDE log-likelihood. Decrease unusable-KDE log-likelihood. Computed as: unusable to usable: usable to unusable:

25. Guiding Latent Space Towards Usability

26. Guiding Latent Space Towards Usability Are Latent Movements Meaningful? Each step in latent space adjusts the bicycle design. KDEs suggest that these steps bring the design closer to regions classified as usable. Question: Does the SVM classifier agree that these designs are moving towards usability? Method: After each latent space move, use the decoder to extract the bike’s parameters. Pass ’CS textfield ’, ’Saddle height’, and ’Stack’ to the SVM and calculate the distance from the decision boundary.

27. Are Latent Movements Meaningful? Unusable to Usable Usable to Unusable Distances from the SVM decision boundary for each latent space move, evaluated on the test set . 10 consistent decrease, 4 slight increase at some point, 2 major increases. The SVM boundary is not the ground truth ; occasional increases in distance may arise from the complex, differing shapes of SVM and KDE decision boundaries. Significant increases might result from the limited training samples or VAE limitations. Overall , each step brings the bike closer to the boundary , with all bikes eventually classified as usable.

28. Guiding Latent Space Towards Usability Are Latent Movements Meaningful? Answer: Yes Overall , each step moves the bike closer to the opposite label for both the usable and unusable bike sets. The agreement between the two models—latent movements guided by KDE bringing bikes closer to the SVM decision boundary— validates these movements. This approach is powerful because: KDEs implicitly capture essential usability traits. While the SVM model requires explicit feature selection for classification.

29. Guiding Latent Space Towards Usability Visualization of Guiding Unusable Designs Towards Usable Regions in the Latent Space: Colored boxes highlight the steps at which the KDE or SVM classifier first marks the design as usable. Blue boxes indicate SVM, red boxes represent KDE. Observation: When KDEs require additional steps for the bike to be classified as usable, only a few steps are needed.

30. Guiding Latent Space Towards Usability Visualization of Guiding Unusable Designs Towards Usable Regions in the Latent Space: Colored boxes highlight the steps at which the KDE or SVM classifier first marks the design as usable. Blue boxes indicate SVM, red boxes represent KDE. Observation: When SVM require additional steps for the bike to be classified as usable, only a many more steps are needed.

31. Guiding Latent Space Towards Usability Reasoning KDEs provide continuous log-likelihood scores across the entire latent space . Some regions may have low scores for both KDEs, indicating low confidence from both models. However, points can still be classified as usable if: p unusable usable 0,000011 0,00001 Introduce Minimum Confidence Threshold Add constraint to ensure higher confidence for usability classification:

32. Guiding Latent Space Towards Usability Threshold Effect on Distance from Regenerated Designs As expected, the threshold has an exponential effect on how much the design changes.

33. Guiding Latent Space Towards Usability Setting High Confidence Threshold of 20: Highlighted in an orange box. Observation: On average, the threshold has a greater impact on the chosen design when the KDE originally requires fewer steps . Brings its classifications closer to the SVM decision boundary in low-confidence regions.

34. Latent Guidance during Interpolation Overview Designers may want to interpolate between two designs. Interpolate while enhancing usability? Method First linearly interpolate between the two bikes in the latent space. Move each latent point towards usable regions guided by the latent KDE model. Introduce a maximum number of iterations to ensure that each latent point is not moved too far. This can be easily adjusted by the designer. How does this look with two unusable bikes?

35. Latent Guidance during Interpolation

36. Latent Guidance during Interpolation Plot showing the distances of each bike from the SVM decision boundary. Positive distances indicate more unusable bikes, while negative distances indicate more usable bikes, with 0 representing the decision boundary. Guided interpolation enhances usability across all interpolated points, even for already usable designs.

37. Latent Guidance during Interpolation

38. Latent Guidance during Interpolation While visual differences are minimal, guided interpolation enhances the usability of 9 out of 11 bikes for both maximum iterations.

39. Research Question 2 RQ2: Given a set of usable/unusable bicycles, can a generative model be guided to create new designs or improve existing ones to reflect the desired quality? Sample usable bikes using latent KDEs. Minimum confidence increasing chance of usable design. KDEs effective in enhancing usability. Designers can control the extent of modifications . Ultimately should be confirmed by human evaluators.

40. Guiding Latent Space Impact Latent guidance provides a method for human-AI collaboration. KDEs implicitly capture parameter relationship with usability. Provide flexibility to control the number of adjustments through hyperparameters (e.g., minimum confidence, maximum iterations). Possibility of custom criteria , such as SVM or parameter constraints , defining precise stopping conditions for design refinement.

41. Guiding Latent Space Limitations of the Study Bicycle ratings are subject to the quality of responses from the survey participants. Possibility of unidentified parameter relationship due to a small set of 100 bicycles. The VAE struggles to capture fine details of bicycle designs. Fitting the KDEs is heavily dependent on the performance of the encoder and decoder. Difficult to distinguish if design variations stem from guidance or VAE limitations. Limited interpretability of the latent space.

42. Guiding Latent Space Future Work: Latent Guidance Improve training of VAE to better capture the training set of the KDEs. Investigate KDE-guided interpolation between a source bike and multiple usable target bikes to give designers multiple ways of improving usability. Implement real-time feedback in the design process for continuous improvement based on designer preferences. Explore alternative deep generative models with latent space (e.g., GANs).

43. Guiding Latent Space Future Work: Multi-Objective Counterfactuals for Design (MCD) Automates and simplifies counterfactual search for tailored design recommendations. Allows surrogate models to evaluate a design candidate's performance. Surrogate Models: Predictive models (e.g., regression, classification) can be trained on rated bicycles. High-accuracy SVM can be used to validate generated designs. MCD: Source