About collecting custom fine tuning data

[JaehongJaehongMin]

Hello!
First of all, thank you for sharing your great work!
I was impressed that CogACT performs well on SimplerEnv and LIBERO simulation, even without additional fine-tuning, unlike some other models.

I’m planning to deploy CogACT on a UR10e robot arm in our lab, as it showed the best performance among the models I've reproduced. I read your advice in the README that collecting our own fine-tuning dataset is recommended for deployment in a new environment.

I have one question regarding that:
Do I need to align the UR10e's coordinate axes with a specific convention? Or is it enough to just ensure that the relative alignment between the robot and the camera is consistent between fine-tuning and deployment?

Thank you in advance!

[arnoldland]

Many thanks for your recognition and the efforts invested in reproducing the results!
The discussion here involves two kinds of demands:

1.​​ ensuring the satisfactory task performance​​.
For a new robot and environment, we observed that maintaining a fixed relative position and orientation between the camera and the robot while aligning actions into the camera coordinate system during both training and prediction phases already yields satisfactory performance. The camera placement can be roughly configured in a first-person perspective similar to Google Robot (numerical precision is unnecessary), as demonstrated in ​​our Appendix Figure I​​. Notably, even with deviations in camera angles (e.g., the configuration in ​​our Appendix Figure II​​), the system remains robust.
2. exploration of the optimal camera and robot configuration​​.
Achieving optimal performance requires considering additional factors: leveraging pre-trained data effectively, minimizing occlusions in camera-robot positioning, etc. If the target robot already has sufficient instances in the pre-training dataset, aligning configurations with those in the training data is preferable.

I understand your primary focus is on ​​Point 1​​, which means you just need ensuring consistent relative positioning between the robot and camera during both fine-tuning and evaluation phases. This alignment is critical for maintaining action-space consistency across training and deployment environments

[JaehongJaehongMin]

Thank you very much! Your advice has been really helpful.
May I ask one more question?

I saw your comment in another issue mentioning that CogACT uses Euler angles (Is it aligned with world frame or tcp frame?).
However, I believe our UR robot operates based on axis-angle representation.
In that case, should I convert the Euler angle output from CogACT into axis-angle before sending commands to the robot arm?

I assume CogACT learns better from real-world fine-tuning data using Euler angles, so I'm planning to collect the data in Euler format.
Does this sound like the right approach?

[arnoldland]

1. Q: Is it aligned with world frame or tcp frame?
   Our Euler angles are represented in a fixed coordinate system, which can be considered closer to a world frame. In practice, both the fine-tuning data and testing for our real-world robots involve converting all actions to a fixed camera coordinate system.

2. Q: Does this sound like the right approach?
   Yes, you are right. We recommend first converting the axis-angle representation training data to Euler angles for fine-tuning CogACT, then converting the Euler angle rotations output by CogACT back to axis-angle representation. This approach is intended to ensure unified representation of the model's output across different robots, while actual deployment only requires conversion to the specific control mode.

[JaehongJaehongMin]

I saw that you’ve converted all actions to a fixed camera coordinate system — that’s great to know, I’ll keep that in mind going forward.

Also, I believe training CogACT from scratch (using OpenVLA or LAMA2 for initialization) would be a valuable experience for me, especially given its strong performance and the clear guidance you've provided.

I’ve downloaded and preprocessed the OXE dataset using the code from the OpenVLA repository (https://github.com/moojink/rlds_dataset_mod/blob/main/prepare_open_x.sh). Most of the datasets seem to be processed correctly, but I noticed something unusual with the FMB dataset. According to the official OXE documentation, the FMB dataset should be around 360GB. However, the version I downloaded is over 1TB — nearly four times larger than expected.

Could you kindly let me know the actual size of the FMB dataset you used during pretraining CogACT?

[arnoldland]

The FMB dataset we downloaded is indeed 356GB. Perhaps you can check if there has been a version update of the FMB dataset that caused the increase in size. However, I checked the original FMB dataset repository, and the reported total size is still less than 1TB.

[JaehongJaehongMin]

Thank you very much for your kind explanation.
I'm sorry for asking so many questions. I'm still quite new to the VLA field.

Before collecting real-robot trajectories, I'm trying to gather trajectories in the SimplerEnv simulation environment, which is mentioned in your paper.
I collected 25 trajectories for each of the 4 tasks (Carrot, Cube, Spoon, Eggplant) in SimplerEnv using CogACT-Base, trying to ensure as much diversity in episodes as possible.
The trajectories contain images (480*640 rgb) and actions (7-DoF, rotations are in Euler format).
Specifically, I collected 'raw_actions' at this line SimplerEnv
I then converted the collected trajectories to the RLDS format, following this repository: RLDS format converter.
Finally, I fine-tuned CogACT-BASE with the collected trajectories (converted to RLDS format). I'll refer to this as my data.
However, the success rates were lower than default CogACT-Base (sadly...).

Meanwhile, I obtained another fine-tuning dataset from a different repository (I'll call this other_repo data) that was used to fine-tune OpenVLA.
It worked well with OpenVLA, so I fine-tuned CogACT with this other_repo data, and the performance improved.

I analized the other_repo dataset, and I found that they qunatized the action space (possibly for tokenization?).
So I applied quantization to my data as well and fine-tuned CogACT. But the success rates were still lower than default CogACT-BASE.
I think I may have made mistake when preprocessing the data (quantizing the action space of my data).

Long story short, my questions is that 'is it okay that just using raw action output for fine-tuning data?'.
If not, should I apply some other preprocessing?

I apologize for the inconvenience, but your assistance would be greatly appreciated. Thank you very much for your time and support.

[JaehongJaehongMin]

I'm attaching the histogram of the action distributions (from left, each graph corresponds to delta x, delta y, delta z, delta row, delta pitch, delta yaw, gripper width) of my data (not quantized) and other_repo data.
The first row is my data and the second row is other_repo data.

You can find out that the actions in other_repo data are quantized (I checked that many action values are dulicated, seldom happens in continuous space).

[arnoldland]

It's OK to discuss this here!
First of all, I don't think quantized data is necessary for CogACT, because diffusion models are able to model continuous distributions (but you need to normalize data).
Secondly, has the model been fine-tuned for a sufficient number of steps? The loss in diffusion models typically drops rapidly at first, followed by a long period of slow decline. So even if the diffusion loss appears to decrease slowly over a short period, it doesn't necessarily mean the training is sufficient (for example, training for just a few thousand steps may not be enough).
Additionally, if there's concern about the quality of the collected data, you can replay the collected ground truth data in the simulation environment to verify whether the task can still be completed.

[JaehongJaehongMin]

I'm really thankful for your reply. It helps me a lot.

First of all, I don't think quantized data is necessary for CogACT, because diffusion models are able to model continuous distributions (but you need to normalize data).

I admit with you! CogACT can predict action in continuous space, which means doesn't need to quantize it.
Actually, I didn't normalized the actions, because the other_repo data shows good performance even if they are not normalized.
Do you have any idea with it that other_repo data shows good performance without normalization?
By the way, I think I need to try training with my data after normalization.
Is it enough to normalize the actions for each dimension from q01 to q99 with clipping the outlier (outside the q01 and q99)?
(And I guess I should save the statistics before normalization for inference in simulation, because they use very small number for actions. It seems that they ignore the big number, for example bigger than 0.5)

Secondly, has the model been fine-tuned for a sufficient number of steps?

I've trained it for 5000 steps with 128 global batches (16 batches per gpu for 8 gpus) about 200 epochs with 2e-5 lr.
I'm attaching my loss curve below. The pink one is my data (collected with CogACT-Base, not quantized, poor performance), and the purple one is other_repo data (quantized, good performance).
other_repo data show good performance even with 5000 steps (about 200 epochs).
But I should try more steps, because it could be easy for CogACT to learn quantized actions with smaller steps.

Additionally, if there's concern about the quality of the collected data, you can replay the collected ground truth data in the simulation environment to verify whether the task can still be completed.

I've already checked that my data and other_repo data works well in replay, but I think it's a really important point! Thank you very much.

Thank you for all of your advice. I'll try training CogACT with normalized (but save the original statistics like q01 and q99 for inference in Simpler) and more steps!

[arnoldland]

Finetuning the entire network with only 5K steps is indeed too few — a longer finetuning process (at least 10K steps) is needed, and you’ll observe that the loss is still decreasing.

CogACT-Base itself is capable of handling tasks on simpler. Therefore, if you finetune directly with few steps and without normalization, it might interfere with actions that would be correct in the normalized space, potentially leading to performance degradation. Moreover, such a distribution of values may be unfavorable for the diffusion to learn effectively.

During training, you need to save the statistics and restore the predictions from the normalized values when evaluating in the simulation environment. I’m not quite sure why the model seems to perform better after quantization, but according to what you previously mentioned, quantizing your collected data did not lead to better performance. Perhaps the performance gain in the other repo comes from different factors? For example, is there any special technique in how they collected their data?

[JaehongJaehongMin]

Sorry, the quantized at the end of my previous message was a typo — I meant normalized (I’ve fixed it).
I’ll try training with normalization.

a longer finetuning process (at least 10K steps) is needed

I see — I should try with more steps!

During training, you need to save the statistics and restore the predictions from the normalized values when evaluating in the simulation environment.

That was exactly what I was concerned about. Thank you!

Is there any special technique in how they collected their data?

I’ve been trying to find any related information, but haven’t had much luck. I’ve been struggling with this for almost two months. 😢

[jasonjabbour]

@JaehongJaehongMin I saw you mentioned that CogAct performed well on LIBERO without any finetuning. Could you share more about that? I tried running CogAct-Base and CogAct-Large on LIBERO Spatail and LIBERO Goal but it was not successful at completing the tasks

Here is a video of CogAct-Base without any additional finetuning. Here I choose to use Action 0 from the Action chunk. It's getting pretty close but still it is not successful:
https://github.com/user-attachments/assets/566335f5-144f-4aa0-bae5-798640af0788
https://github.com/user-attachments/assets/eb5bd1e0-06ce-4053-8802-34d3adacb922

I also tried Ensemble and using the full Action chunk but those did not make much of a difference. I also tried finetuning (one model for 10k steps total) on all four LIBERO datasets (spatial, object, goal, and 10) but I ended up getting even worse performance:

https://github.com/user-attachments/assets/5d29d346-57ec-46ad-929c-6a64224fa2e7
https://github.com/user-attachments/assets/b3c9e735-1fad-43a6-83b6-04582d2b3b3e

@arnoldland Any advice would be much appreciated!