Understand the dataset structure

[GiulioRomualdi]

Hi @mrsp, first of all thank you for developing gem2. I was trying to train the framework with a custom dataset.

In order to organize data in the same format of the example provided in the repo I was wondering the exact meaning of the file names stored in https://github.com/mrsp/gem2/tree/main/GEM2_nao_training and loaded by the following function

gem2/src/gem2_tools.py

Lines 102 to 169 in af30917

	rfX = np.loadtxt(training_path+'/rfX.txt')
	rfY = np.loadtxt(training_path+'/rfY.txt')
	rfZ = np.loadtxt(training_path+'/rfZ.txt')
	rtX = np.loadtxt(training_path+'/rtX.txt')
	rtY = np.loadtxt(training_path+'/rtY.txt')
	rtZ = np.loadtxt(training_path+'/rtZ.txt')
	lfX = np.loadtxt(training_path+'/lfX.txt')
	lfY = np.loadtxt(training_path+'/lfY.txt')
	lfZ = np.loadtxt(training_path+'/lfZ.txt')
	ltX = np.loadtxt(training_path+'/ltX.txt')
	ltY = np.loadtxt(training_path+'/ltY.txt')
	ltZ = np.loadtxt(training_path+'/ltZ.txt')
	dlen = min(np.size(lfZ),np.size(rfZ))
	gX = np.loadtxt(training_path+'/gX.txt')
	gY = np.loadtxt(training_path+'/gY.txt')
	gZ = np.loadtxt(training_path+'/gZ.txt')
	accX = np.loadtxt(training_path+'/accX.txt')
	accY = np.loadtxt(training_path+'/accY.txt')
	accZ = np.loadtxt(training_path+'/accZ.txt')
	dlen = min(dlen,np.size(accZ))
	dcX = np.loadtxt(training_path+'/comvX.txt')
	dcY = np.loadtxt(training_path+'/comvY.txt')
	dcZ = np.loadtxt(training_path+'/comvZ.txt')
	dlen = min(dlen,np.size(dcZ))
	if(self.gem2):
	lvX = np.loadtxt(training_path+'/lvX.txt')
	lvY = np.loadtxt(training_path+'/lvY.txt')
	lvZ = np.loadtxt(training_path+'/lvZ.txt')
	dlen = min(dlen,np.size(lvZ))
	rvX = np.loadtxt(training_path+'/rvX.txt')
	rvY = np.loadtxt(training_path+'/rvY.txt')
	rvZ = np.loadtxt(training_path+'/rvZ.txt')
	dlen = min(dlen,np.size(rvZ))
	lwX = np.loadtxt(training_path+'/lwX.txt')
	lwY = np.loadtxt(training_path+'/lwY.txt')
	lwZ = np.loadtxt(training_path+'/lwZ.txt')
	rwX = np.loadtxt(training_path+'/rwX.txt')
	rwY = np.loadtxt(training_path+'/rwY.txt')
	rwZ = np.loadtxt(training_path+'/rwZ.txt')
	#laccX = np.loadtxt(training_path+'/laccX.txt')
	#laccY = np.loadtxt(training_path+'/laccY.txt')
	#laccZ = np.loadtxt(training_path+'/laccZ.txt')
	#dlen = min(dlen,np.size(laccZ))
	#raccX = np.loadtxt(training_path+'/raccX.txt')
	#raccY = np.loadtxt(training_path+'/raccY.txt')
	#raccZ = np.loadtxt(training_path+'/raccZ.txt')
	#dlen = min(dlen,np.size(raccZ))
	baccX_LL = np.loadtxt(training_path+'/baccX_LL.txt')
	baccY_LL = np.loadtxt(training_path+'/baccY_LL.txt')
	baccZ_LL = np.loadtxt(training_path+'/baccZ_LL.txt')
	baccX_RL = np.loadtxt(training_path+'/baccX_RL.txt')
	baccY_RL = np.loadtxt(training_path+'/baccY_RL.txt')
	baccZ_RL = np.loadtxt(training_path+'/baccZ_RL.txt')
	baccX = np.loadtxt(training_path+'/baccX.txt')
	baccY = np.loadtxt(training_path+'/baccY.txt')
	baccZ = np.loadtxt(training_path+'/baccZ.txt')
	bgX_LL = np.loadtxt(training_path+'/bgX_LL.txt')
	bgY_LL = np.loadtxt(training_path+'/bgY_LL.txt')
	bgZ_LL = np.loadtxt(training_path+'/bgZ_LL.txt')
	bgX_RL = np.loadtxt(training_path+'/bgX_RL.txt')
	bgY_RL = np.loadtxt(training_path+'/bgY_RL.txt')
	bgZ_RL = np.loadtxt(training_path+'/bgZ_RL.txt')
	bgX = np.loadtxt(training_path+'/bgX.txt')
	bgY = np.loadtxt(training_path+'/bgY.txt')
	bgZ = np.loadtxt(training_path+'/bgZ.txt')
	dlen = min(dlen,min(np.size(bgZ_LL),np.size(baccZ_RL)))

cc @paolo-viceconte

[mrsp]

Hello @GiulioRomualdi

Initially I suggest running the GEM1.0 version which is an unsupervised learning gait phase detector implementation details are in https://ieeexplore.ieee.org/document/8793598

To do that you need the following:

• rf/rt is the right leg force/torque as measured by Force/Torque sensors in the legs or computed with force sensitive resistors.
• g/acc is the gyro/accelerometer measurements in the base frame
• dc is the CoM velocity computed with kinematics in the base frame e.g. you can compute the CoM position w.r.t to the base every time instant and differentiate this.

To run GEM2.0 you need have to IMUs mounted on the feet. In that case GEM uses supervised learning to learn the contact and gait-phase probabilities.

I have also contributed this pkg https://github.com/mrsp/gem2_state_publisher to assist in the data collection process.
Moreover, I have created this MATLAB script to create the dataset files from a ros bag file: https://github.com/mrsp/gem2/blob/main/src/MATLAB/rosbagToGEM2v2.m

As it is right now all quantities must be in a common frame of reference. The MATLAB file aligns all vectors in the world frame with the base IMU orientation. (Nevertheless it has worked well with just the raw measurements).

Let me know if this helps you.

[GiulioRomualdi]

Thanks for the prompt answer. We (@paolo-viceconte) will try to follow your suggestions.

[mrsp]

Sure, also kindly let me know how it goes and if there are things/features to improve by a user perspective.

The gem2 pkg is an open research work on progress with more features to come in the near future due to contact detection being (in my opinion) one of the most important topics that must be accurately addressed in order to achieve agile legged locomotion.