Mainframe & FICON

Reply
Occasional Contributor
Posts: 5
Registered: ‎03-18-2011

Question on ISL/Trunk Master usage : Why isn't Port utilisation more evenly balanced?

Hi,

Firstly some background.

We have 2 sites and 2 pairs of dedicated (for Ficon) 48K's each of which has two trunk groups to its reciprocal 48k at the other site (Via dwdm)

Each trunk group has 4 ports which are 4GB.

portcfglongdistance is LS (long static) mode.

Fabric o/s is V6.4.2a

We have dynamic path selection as shown by the D in the topologyshow output here:-

PORCD001:admin> topologyshow

2 domain(s) in the fabric; Local Domain ID: 200

Domain:         201
Metric:         500
Name:           HAVCD001
Path Count:     2

        Hops:                   1
        Out Port:               2/16
        In Ports:               2/4 2/5 2/11 2/12 2/13 1/24 2/20 2/21 2/22 2/23 2/24 2/25
        Total Bandwidth:        16.000 Gbps
        Bandwidth Demand:       287 %
        Flags:                  D

        Hops:                   1
        Out Port:               1/16
        In Ports:               1/4 1/5 1/8 1/9 1/10 1/11 1/12 1/13 2/8 2/9 2/10 1/20 1/21 1/22 1/23 1/25

        Total Bandwidth:        16.000 Gbps
        Bandwidth Demand:       312 %
        Flags:                  D

So i would expect the load to be balanced between both trunk groups.

However, The Master port on each trunkgroup seems to get most of the workload, where i would expect to see a more even flow across all 4 ports on each group.

Output below is from the porterrshow for frames tx & rx for about the last 18 hours.

  

128:  101.1m   1.1g
129:  572.0k  29
130:   17.0m  37.1k
131:  367.7m   9.9m

144:    1.9g   2.4g
145:   20.1k   2.2m
146:    5.2m  65.4m
147:  411.6m   1.1g

PORCD001:admin> trunkshow
  1:128->128 10:00:00:05:1e:36:36:76 201 deskew 16 MASTER
    130->130 10:00:00:05:1e:36:36:76 201 deskew 18
    129->129 10:00:00:05:1e:36:36:76 201 deskew 16
    131->131 10:00:00:05:1e:36:36:76 201 deskew 15

  2:144->144 10:00:00:05:1e:36:36:76 201 deskew 16 MASTER
    147->147 10:00:00:05:1e:36:36:76 201 deskew 16
    146->146 10:00:00:05:1e:36:36:76 201 deskew 15
    145->145 10:00:00:05:1e:36:36:76 201 deskew 17

We are getting tim_rdy_pri & tim_txcrd_z on the ISL's so am in the process of increasing the number of buffer credits available.  

It seems a bit pointless to ramp up buffer credits across all 4 ports per trunk group if the Master is going to hog most of the load.

Why isnt the port usage per trunk group more even?

i expect i'm missing something but it's not leaping out at me from the manuals! 

Any input gratefully received.

Thanks

External Moderator
Posts: 4,788
Registered: ‎02-23-2004

Re: Question on ISL/Trunk Master usage : Why isn't Port utilisation more evenly balanced?

shaf rafiq,

--->>>The Master port on each trunkgroup seems to get most of the workload,....

---->>>.....where i would expect to see a more even flow across all 4 ports on each group.

I facing the same issue around 6 Mouth ago, and this is official answer from Brocade Third Level Support.

This is a normal behavior, because DPS in some case by Long Distance, is not able to "handle/balancing" the Traffic.

TechHelp24
Occasional Contributor
Posts: 5
Registered: ‎03-18-2011

Re: Question on ISL/Trunk Master usage : Why isn't Port utilisation more evenly balanced?

Forgot to mention that the Aptpolicy is 1 for port based routing.

Thanks

Highlighted
New Contributor
Posts: 4
Registered: ‎09-23-2009

Re: Question on ISL/Trunk Master usage : Why isn't Port utilisation more evenly balanced?

Remember that the output in porterrshow (frames tx/rx) is based on 32-bit counters (wrap after 4.2g). (you can also use portstats64show x/x output that is a 64-bit counter).

The load-balancing algorithm within a trunk has always done it this way since the 4Gbps platforms and the “master” assignment has nothing to do with the load-balancing of frames.

Not until the first ISL port is fully utilized the second port will be used and so on… (is done in a pre-determined way based on internal bport numbers).

You should configure the same settings across all ports within the trunk and as its FICON, I would at least set the LS distance to about 3x the actual distance to compensate for less frame size (more buffer credits).

--filiph

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.