Hakim al Umma Mawlana Ashraf Ali Thanawi (Allah have mercy on him) said,

‘The bayiah from women should not be done by taking their hand in one’s hand.

The Prophet (Allah bless him and give him peace) never touched a woman while taking bayiah.

It is impermissible (haram) to touch an unrelated (non mehram) woman.

The author of ‘Mehboob as Salikeen‘ writes,

‘The bayiah of women is as follows,
If they are not present at the occasion than they become bayiah via representation of a mehram relative. The instructions (for sulook) are given to them via this representative. The Shaikh (may) give them a piece of cloth from his garments (as a token for this pact).

If the women are present at the occasion than they become bayiah from behind a veil. The hand holding should not be done.
Hence, the Shaikh takes allegiance from men by holding their hand but not from the women.
Moreover, it is mentioned in this book that the exclamation ‘I accept’ (at the time of acceptance of allegiance) is for men (only).
Then the Shaikh instructs the women to do good and abstain from sinful (from behind the veil).

Taleemud Deen, page 192-3 (Rehmania edition)