To roll your own layout, choose an interval for each direction (South-east and South-west) and a choose whether to use Horizontal or Vertical Layout.
Left-to-right: South-west interval, South-east interval, Orientation.
You can also choose the note size:
All hexagonal layouts are defined by the intervals in each of the six directions. For a layout to be isomorphic
(making patterns the same in all keys), an interval in one direction must be the same for each note, meaning that regardless of where you start, going south-east will always be, say, a major fifth, and therefore going the opposite direction (north-west) will always be the opposite interval (a descending major fifth, in this case)
So while it looks like you can define six intervals (one for each direction), three of the directions are constrained to be the opposite interval of the opposing direction, and thus can't be defined. But if you look a little deeper, you can only define two intervals.
If you go south-east, and then south-west, and then north, you end up back at the same note, making those three notes a triangle. If you know two of those intervals, the third must take you back to the original note for the keyboard to stay isomorphic. So really, all you need to do is choose two intervals to define the layout. The third is the sum of the other two, in the opposing direction.
For example, in the Harmonic layout, the south-east direction is a descending minor third (3 semitones), and the south-west direction is a descending major third (4 semitones). the sum of these intervals is seven semitones (a fifth), so the north direction (the third leg in the triangle) must be an ascending fifth.
In Musix Pro, you can define the interval for the south-east direction and the south-west direction, and all other intervals will be calculated and filled in. For example, for the wicki-hayden layout, south-east is a descending perfect fourth, and south-west is a descending perfect fifth.
Define South-east Interval
Define South-west Interval
One more feature to choose is whether the layout is horizontal or vertical. Horizontal layouts have an east-west direction but no north-south direction, and vertical layouts have a north-south direction but no east-west direction.
Select Layout Orientation