Clothing size recommender on real-time fitting simulation using skeleton tracking and rigging

– Virtual fitting room (VFR) is a technology that replaces conventional fitting rooms. The VFR is not only available in shops, malls, and any shopping center but also in online stores, which makes VFR technology more and more developed, primarily to support online garment sales. VFR become a trending research interest since Microsoft has developed a Kinect tracking system. In this paper, we proposed the interactive 3D virtual fitting room using Microsoft's Kinect tracking and the rigging technique from 3D Modeling Blender and to implement the VFR. VFR manages the progress of virtual fitting that forms the three-dimensional simulations and visualization of garments on virtual counterparts of the real prospective buyer (user). Users can view the clothing animation on the various poses that are following the user body movements. The system can evaluate the user’s match, guiding them to choose the suitable size of the clothes using Euclidean distance.


I. INTRODUCTION
Virtual fitting room (VFR) is a new technology that facilitates prospective buyers (users) to try several clothes virtually. In this VFR technology, users do not need to physically place their bodies to choose the size, color, style, and other parameters related to buyer satisfaction [1]. In the last few decades, several studies have been applied to develop a virtual fitting room (VFR) due to the tremendous popularity of virtual reality and augmented reality in the garment industry.
The existence of new technologies such as Kinect, a powerful web camera, and a smartphone makes it possible to learn new systems for trying on clothes without doing it physically at the store [2]. VFR implementation in several existing studies and which have been implemented in official stores not only use 2D but also in 3D to display their virtual clothing [3]- [6].
Examples of the application of 2D / 3D in VFR technology, one of which is implemented by Zugara.
The use of 2D or 3D technology is not a problem for users, but 3D looks more refreshing and is more fun for users. There is comparable to the development costs incurred higher than 2D development. The experience of enjoying VFR is far more essential to prioritize [7].
VFR technology in the last decade, not only available in stores, malls, and shopping centers, but also online stores. This technology makes users comfortably fitting the clothes in real-time without becoming complicated and having to take off their clothes [8]. Especially for online stores whose clothes are physically not available, It can reduce the risk of returning sales because of unsuitable size. For the seller, this will increase sales in the e-commerce perspective [9].
Kinect is one of the most popular technologies used in implementing VFR. The calibration process is carried out by Microsoft Kinect using a 3D depth sensor and an RGB camera makes this VFR application real-time following human movements using a humanoid model [10], and can also be used as a 3D virtual dressing room system [11]. Using Kinect, the user does not need to use any additional control devices to track users' positions, poses, and gestures [12].
Rigging is the process of positioning a frame obtained from a Kinect camera to a human model. With this rigging technique, virtual clothes can be adaptive to the user's movements [13]. Closely related to VFR technology is to recommend the size of the virtual clothes to be tried, one method for measuring clothing sizes can use the Euclidean distance technique [14].
This research proposes a virtual fitting room by combining the concept of 3D virtual cloth, user motion pose adaptation, and cloth size recommender. Unlike [10], [11] which used 3D and humanoid avatars to make it more natural, this study used no virtual avatars, but the original user's body. This study applied the rigging technique from 3D Blender like [4], [12] to simulate virtual clothes that can follow the movements of the consumer's body. To find out the coordinates of the location of 3D objects using the Kinect calibration process and finally to recommend clothing size, this system used Euclidean distance like [14].
VFR addresses problems when using traditional fitting rooms in stores or malls, such as queuing to try on clothes, having to take off cloth when trying new ones, and the amount of clothing that can be brought to Copyright  room fittings. It also can overcome the problem of mismatching the size, color, and model of clothes when purchasing online. The hope with this VFR technology, users can try clothes without these problems, and users can also move right / left as if trying on clothes in front of a mirror, but the virtual clothes worn are still attached to the body.

A. Making a 3D object and adding a rigging technique
The creation of the 3D objects used a 3D blender modeling software to simulate virtual clothes which will later be attached to the body of the user. The results of the 3D modeling of virtual batik clothing shown in Figure 1. The rigging technique applied to add armature (metarig) to 3D objects. This method recognizes virtual clothing to move according to the movements of users. The addition of rigging techniques to virtual clothes can be seen in Figure 2.

B. Calibration process
The calibration process was carried out by Microsoft Kinect using a 3D depth sensor and an RGB camera. This calibration process produced tracking 3D objects. This method causes Kinect can detect objects that are within the Kinect range. One of the results of 3D Object Tracking is the skeleton data, as shown in Figure 3. This research used only three points of skeleton data, which were skeleton_shoulder_right, skeleton_shoulder_center, and skeleton_shoulder_left. From these three points, the system was able to put virtual clothes on the user's body.

C. Clothing size recommendation
The first step for clothing size recommendation was calculating the distance between skeleton_shoulder_right and skeleton_shoulder_left. Vector distance between the two vectors uses Euclidean distance. The distance implemented the Vector3.Distance function in Unity. The next step was comparing these results with the standards of Indonesian clothing size as the basis of the system to provide clothing size recommendations according to Table 1.

D. Addition of 3D clothing objects to the prospective buyer body
The addition of 3D clothing was done when users choose the desired clothing. Virtual 3D clothes will be added to the skeleton_shoulder_center, skeleton_shoulder_right, and skeleton_shoulder_left coordinates of users.

III. RESULTS AND DISCUSSION
This experiment was conducted to determine the value of the accuracy of clothing size recommendations given by the system. The accuracy measurement used variations in the distance between users with Kinect sensors and Kinect height, and the influence of poses made by users on the recommended clothing sizes provided by the system. This experiment was implemented with five conditions.

A. Clothing size recommendations with variations in distance and height from Kinect
In this step, analysis and experiment were carried out to determine the level of system accuracy. The analysis variables used are Kinect height (Kinect placement distance measured from the flat ground) and the distance between users and Kinect sensors. Kinect height is expressed into three measures, namely 55 cm, 65 cm, and 75 cm. The distance of users with Kinect sensors is distinguished by a distance of 200 cm, 225 cm, and 250 cm.
This experiment was tested directly on 23 users consisting of 13 women and 10 men who were randomly selected. A total of 117 trials were conducted for women's clothing and 90 trials for men's clothing. Details of the analysis results can be seen in Table 2 and  Table 3, while Table 4 shows the recapitulation results summarized from both tables.
It can be concluded that the Euclidean distance method can be used to measure the user's back distance. The Euclidean distance between skeleton_shoulder_right and skeleton_shoulder_left can be used as the basis for determining the recommended clothing size as [14]. The average accuracy rate of this system is 81 % for males and 69 % for females, as shown in Table 4.
The best accuracy for this scenario is achieved with a Kinect placement height of 55 cm for female users and of 75 cm for male users with a distance of 200 cm between the user and Kinect. This corresponds to [8] which recommends that the optimal placement point for Kinect height be 55 cm and 75 cm with the distance between the user and Kinect is 200 cm.

B. Simulation of virtual clothes in real-time according to the motion of users
This analysis and experiment were carried out to arrange the position of 3D virtual clothes to match the user's body position and to simulate the movement of virtual clothes following body movements in real-time. Figure 4 shows clothes attached to the body of a user, while Figure 5 is a real-time simulation of virtual clothes following the user's body movements. These results are in accordance with the previous studies in [4], [13]. Copyright Table 5 shows the result of an analysis and experiment conducted to determine the influence of users' poses on system size recommendations. Five poses are done; there are stand up straight (1), stretching hands (2), hands forward (3), stand oblique to the right (4), and stand oblique to the left (5).

C. Clothing size recommendations with user's poses variation
Various poses cause Kinect calibration reference points (skeleton_shoulder_right and skeleton_shoulder_left) are also different, thus also affecting recommended clothing size recommendations. The best pose for recommended clothing size is the first pose (stand up straight). The virtual clothes can adapt to the user's movements according to [4], although the results of the recommendation of clothing sizes with different poses make the recommended size unstable.

D. The effects of users moving forward and backward from Kinect
This analysis and testing were conducted to determine the effect of Kinect distance on the performance of this virtual clothing. In general, the image will look smaller if it is far from the mirror view and vice versa. This experiment shows how virtual clothes are simulated when the user is at a distance of 200 cm, 225 cm, and 250 cm. The results of this experiment can be seen in Figure 6, where when the user stays away from Kinect, virtual clothes become smaller. From Table 5, it can be concluded that the Kinect distance slightly affects the clothing size recommendations for the best pose 1.

E. Accuracy based on user height
This analysis and experiment were conducted to discover the effect of the user's height in determining clothing size recommendations. This experiment was      Table 6, it can be concluded that the height of the user does not affect the clothing size recommendations. The height of the user does not affect the recommended clothing size as long as the whole body of the user is in the Kinect coverage area.
From all of these experiments, making 3D clothing objects with rigging in the form of adding armature to 3D clothing objects can make virtual clothing adaptable to the movements of users in real-time. When users raise their hands, the arms on the clothes also lift. Virtual clothing simulations can follow the motion of users and always stick precisely to the user body. The Euclidean distance method in this system can provide clothing size recommendations to users with an average accuracy rate of 81% for male and 69% for female using a variety of Kinect heights (55 cm, 65 cm, and 75 cm) and variations in the Kinect distance with users ranging from 200 cm, 225 cm and 250 cm. The recommended size of clothes is influenced by the height of the Kinect, the distance between the user and the Kinect, the pose of the user. The user's height does not affect the outcome of clothing size recommendations.
The VFR technology proposed in this paper can simulate the movement of clothing following the user's body pose almost the same as previous studies using Kinect in [4], [12] and not replacing people with avatars [2], [13]. However, this research is limited to virtual clothing without accessories other supporters, unlike previous studies that use various types of complementary accessories [5]. This study also produced recommendations about clothing sizes using the same techniques as in [14], but not as detailed as measuring clothing size patterns as revealed in [6].

IV. CONCLUSION
The proposed virtual fitting room can simulate the user's motion with rigging technology in Blender 3D modeling with the addition of metarig. The virtual clothing can follow the user's movements (such as the user raises his hands up, then the arms of the clothes also lift up). The calibration process provided by Microsoft Kinect makes virtual clothes exactly on the user's body. The clothing size recommendations generated by the system using the Euclidian distance method has an average accuracy rate of 81% for male and 69% for female. The height of the Kinect influences the results of these recommendations, the distance between the Kinect and the user, and the user's pose, while the height of the user does not affect the results of the clothing size recommendation.