联邦学习心得

不定时更新, 该时间段学到、想到的联邦学习知识

arxiv downloader

wget -U NoSuchBrowser/1.0 https://arxiv.org/pdf/你需要下载的论文.pdf

联邦学习现状、关键问题、发展方向

1 问题定义

1.1 典型目标函数

典型联邦学习问题:

数百万远程设备上学习单个全局模型。

约束: 设备生成的数据被本地存储和处理，只有中间更新周期性地与中央服务器进行通信。

因此目标通常是最小化以下目标函数：

$ \min_{w} F(w), \text { where } F(w):=\sum_{k=1}^{m} p_{k} F_{k}(w) $
m 是设备总数， $ p_k ≥ 0 $ 并且 $ ∑_kp_k = 1 $
$ F_k $(w) 是第 k 个设备的本地目标函数（一般是个loss损失函数）

例如：$F_{k}(w)=\frac{1}{n_{k}} \sum_{j_{k}=1}^{n_{k}} f_{j_{k}}\left(w ; x_{j_{k}}, y_{j_{k}}\right)$，表示第k个客户端上，所有训练样本的损失值取平均

一般, 第k 个客户端的$ p_k $ ： $ p_k = 1/n $ 或者 $ p_k = n_k/n $ , $ n = ∑_k n_k $ (即，n=所有客户端的训练数据)

2. 核心挑战

2.1 挑战1：昂贵的沟通

为了使模型与联邦网络中的设备生成的数据相匹配，因此有必要开发通信效率高的方法,作为训练过程的一部分

迭代地发送小消息或模型更新，而不是通过网络发送整个数据集

为了减少通信，需要考虑的两个关键方面是：

（i）减少通信回合的总数，

（ii）在每一回合减少发送的消息大小。

2.2 挑战2：系统异构性

硬件（CPU，内存）、网络连接（3G，4G，5G，wifi）和电源（电池电量）的变化，联邦网络中每个设备的存储、计算和通信能力可能不同。

同时活跃的设备通常仅占一小部分，例如，一百万个设备网络中的数百个活跃设备

因此，必须：

(i) 预计参与人数较少

(ii) 容忍异构硬件

(iii) 对网络中的已下线设备具有鲁棒性。

2.3 挑战3：统计异质性

设备经常以non-IID的方式在网络上生成和收集数据，例如，移动电话用户在下一个单词预测任务的上下文中使用了不同的语言

各设备的数据点的数量可能有很大的变化，并且可能存在捕获设备之间的关系及其相关分布的底层结构

2.4 挑战4：隐私问题

联邦学习通过共享模型更新（例如梯度信息）而不是原始数据，朝着保护在每个设备上生成的数据迈出了一步

然而，在整个训练过程中模型更新的通信仍然可以向第三方或中央服务器显示敏感信息。

虽然最近的方法旨在使用安全多方计算或差异隐私等工具增强联邦学习的隐私性，

但这些方法通常以降低模型性能或系统效率为代价提供隐私。

在理论和经验上理解和平衡这些权衡是实现私有联邦学习系统的一个相当大的挑战。

3.解决方案

3.1 高效通信

几个一般的方向，我们将其分为（1）局部本地更新方法，（2）压缩方案和（3）去中心化训练

3.1.1 局部本地更新方法

联邦学习最常用的方法是联邦平均（FedAvg），这是一种基于局部随机梯度下降（SGD）平均更新的方法。

FedAvg在经验上表现得很好，特别是在非凸问题上,但它没有收敛性保证，并且在实际情况下，当数据是异构的时，它可能会发散

FedAVG通过增加了节点本地结算量，减少了通信量

FedSGD VS FedAVG

类型	client	server
FedSGD	1.接受 server下发的全局权重W 2.使用全局权重W和本地数据进行训练，计算出当前梯度g 3. 把客户端的梯度g 发给服务端 （每完成一次训练就发一次）	1.接受 client上送的梯度g1,g2… 2. 计算 全局梯度G = g1 + g2 + … 3. 全局权重W = α*G 4. 把全局权重W广播给所有client
FedAVG	1.接受 server下发的全局权重W 2.使用全局权重W和本地数据进行训练，计算出当前梯度g 3. 客户端的梯度g 更新本地权重 w 4. 重复 执行 2-3 一共 n 次 5. 发送 本地训练 n 次后的 权重w 给server	1.接受 client上送的权重w1,w2… 2. 计算 全局权重 W = ( g1 + g2 + …) / m 3.把全局权重W广播给所有client

3.1.2 压缩消息

虽然本地局部更新方法可以减少通信的总轮数，

但模型压缩方案（如稀疏化、子采样和量化）可以显著减少每轮通信的消息大小。

一些在联邦设置中的实用策略，例如

1）强制更新模型变得稀疏和低秩；

2）使用结构化随机旋转执行量化；

3）使用有损压缩和随机失活来减少服务器到设备的通信；

4）应用Golomb无损编码。

3.1.3 去中心化训练

在联邦学习中，星形网络（中央服务器连接到设备网络）是主要的通信拓扑结构；

去中心化拓扑（其中设备只与它们的邻居通信）作为一种潜在的替代方案。

在数据中心环境中，当在低带宽或高延迟的网络上操作时，去中心化训练被证明比中心化训练更快；

在联邦学习中，去中心化算法理论上可以降低中央服务器上的高通信成本。

最近的一些工作研究了中心化训练。然而，它们要么局限于线性模型，要么假设设备完全参与。

最后，还提出了层级通信模式，以进一步减轻中央服务器的负担，

首先利用边缘服务器聚合来自边缘设备的更新

然后依赖云服务器聚合来自边缘服务器的更新。

虽然这是一种有前途的减少通信的方法，但它不适用于所有网络，因为这种类型的物理层次可能不存在或先验已知。

3.2 系统异质性

关键方向分为：(i) 异步通信，(ii) 主动设备采样，和(iii) 容错。如第2.1.3节所述，我们在下面的讨论中假设了星形拓扑。

3.2.1 异步通信

同步方案在设备变化面前，它们更容易受到掉队者的影响。

异步方案是一种很有吸引力的方法来减轻异构环境中的掉队问题，特别是在共享内存系统中。

然而，异步方案通常依赖于有界延迟假设来控制过时的程度，对于设备k来说，这取决于自设备k从中央服务器拉取到的已更新的其他设备的数量。在联邦设置中，经典的有界延迟假设可能是不现实的，在联邦设置中，延迟可能是小时到天的顺序，或是完全无界的。

3.2.2 主动采样

在联邦网络中，通常只有一小部分设备参与每一轮的训练。然而，绝大多数联邦方法，是被动的，因为它们的目的不是影响哪些设备参与。

另一种方法是在每一轮中积极选择参与设备。目的是让服务器在预定义的时间窗口内聚合尽可能多的设备更新。

同样，激励机制以鼓励具有更高质量数据的设备参与学习过程时，虽然这些方法主要关注系统可变性以执行主动采样，但我们注意到，也值得考虑基于底层统计结构主动采样一组小型但具有足够代表性的设备。

3.2.3 容错

一种实用的策略是简单地忽略这种设备故障，如果故障设备具有特定的数据特性，则可能会在设备采样方案中引入偏差。例如，由于网络连接不良，来自偏远地区的设备可能更容易丢失，因此经过训练的联邦模型将偏向于具有良好网络条件的设备。

编码计算是通过引入算法冗余来容忍设备故障的另一种选择。最近的研究探索了使用代码加速分布式机器学习训练的方法。例如，在存在偏离者的情况下，梯度编码及其变体小心地跨计算节点复制数据块（以及这些数据块上的梯度计算），以获得真实梯度的精确或不精确重构。虽然这对于联邦设置来说似乎是一种很有前途的方法，但是这些方法在联邦网络中面临着根本的挑战，因为由于隐私限制和网络规模的限制，跨设备共享数据/复制通常是不可行的

3.3 统计异质性

3.3.1异构数据Feb框架

通过诸如元学习和多任务学习的方法来对统计异质性进行建模。例如，MOCHA，一个为联邦设置设计的优化框架，可以通过学习每个设备的独立但相关的模型来实现个性化，同时通过多任务学习利用共享的表示。但其扩展到大规模网络的能力有限，且仅限于凸目标。

一种方法将星型拓扑建模为贝叶斯网络，并在学习期间执行变分推理。但推广到大型联邦网络是昂贵的。

在对联邦数据建模时，考虑精度以外的问题（如公平性）可能也很重要。尤其是，天真地求解如（1）中的总损失函数可能隐含地对某些设备有利或不利，因为所学习的模型可能偏向于具有较大数据量的设备，或者（如果相等地加权设备）偏向于通常出现的设备组。

3.3.2 非 IID 数据的收敛保证

当数据在网络中的不同设备上分布不一致时，FedAvg等方法在实践中已经被证明是不同的。

在I.I.D.设置中分析了并行SGD和相关变体，它们使本地更新与FedAvg相似。然而，结果依赖于一个前提，即每个局部解算器是同一随机过程的副本（由于I.I.D.假设），这在典型的联邦设置中不是这样的。

为了了解FedAvg在统计异质环境中的性能，FedProx最近被提出。FedProx对FedAvg方法做了一个小的修改，以确保在理论和实践上的收敛性。

FedProx也可以解释为FedAvg的一个通用的、重新参数化的版本，它在考虑设备之间的系统异构性方面具有实际的影响。

3.4 隐私

隐私问题常常促使人们需要在联邦设置中将每个设备上的原始数据保存在本地。但是，作为训练过程的一部分，

共享其他信息（如模型更新）也可能泄漏敏感的用户信息

3.4.1 机器学习中的隐私

用差分隐私来传递噪声数据草图、用同态加密来操作加密数据、以及安全的功能评估或多方计算。

差分隐私最广泛地使用。简单地说，如果一个输入元素的变化不会导致输出分布的太大差异，那么随机化机制是差异私有的；这意味着不能得出任何关于在学习过程中是否使用特定样本的结论。对于基于梯度的学习方法，一种流行的方法是通过在每次迭代时随机扰动中间输出来应用差分隐私。差别隐私和模型精度之间存在着固有的权衡，因为增加更多的噪声会带来更大的隐私，但可能会严重影响精度。

同态加密目前它应用于有限的设置，例如训练线性模型或仅涉及少数实体

当敏感数据集分布在不同的数据所有者之间时，另一个自然的选择是通过安全功能评估（SFE）或安全多方计算（SMC）来执行隐私保护学习。由此产生的协议可以使多个当事方协作计算商定的函数，而不泄漏任何当事方的输入信息。

然而，这些方法可能不适用于大规模机器学习场景，因为它们会带来大量额外的通信和计算成本。

3.4.2 联邦学习中的隐私

除了提供严格的隐私保证外，还需要开发计算成本低、通信效率高、能够容忍掉线设备的方法，而所有这些都不会过度损害准确性。

尽管联邦学习中有各种各样的隐私定义，但通常它们可以分为两类：全局隐私和局部隐私。

全局隐私：在每一轮生成的模型更新对中央服务器以外的所有不受信任的第三方都是隐私的

本地隐私：进一步要求更新对服务器也是隐私的

引入一个SMC协议来保护单个模型的更新，会产生显著的额外通信成本。中心服务器无法看到任何本地更新，但仍可以在每轮中观察精确的聚合结果。

此外，差分隐私可以与模型压缩技术相结合，以减少通信，同时获得隐私利益

4. 未来发展方向

4.1 极端的通讯方案

众所周知，机器学习的优化方法可以容忍精度的不足；这个错误实际上有助于泛化

联邦学习中有多少交流是必要的，还有待观察

4.2 通信降低和Pareto frontier

特别是，最有用的技术将展示Pareto frontier的改进，在相同的通信预算下，在理想情况下，在广泛的通信/精度剖面上，实现比任何其他方法更高的精度。为了有效地进行神经网络推理，已经进行了类似的综合分析，并且为了以有意义的方式比较用于联邦学习的通信简化技术是必要的。

4.3 新的异步模型

分布式优化中最常研究的两种通信方案是批量同步方法和异步方法（假设延迟是有界的）

4.4 异质性诊断

（i）是否存在简单的诊断以快速确定联邦网络中的异质性水平？

（ii）是否可以开发类似的诊断来量化与系统相关的异质性的数量？

（iii）是否可以利用当前或新的异质性定义来进一步改进联邦优化方法的收敛性？

4.5 细微的隐私限制

在实践中，可能有必要在更细粒度级别上定义隐私。 提供一种较弱的隐私形式，以换取更精确的模型。开发处理混合（设备特定或样本特定）隐私限制的方法是未来工作的一个有趣和持续的方向。

4.6 超越监督学习

迄今为止讨论的方法都是随着监督学习的任务而发展起来的，即他们假设联邦网络中的所有数据都存在标签

实际上，在实际的联邦网络中生成的许多数据可能是未标记或弱标记的。此外，目前的问题可能不是将模型与（1）中所示的数据拟合，而是执行一些探索性数据分析、确定聚合统计数据或运行更复杂的任务，如强化学习。在联邦网络中解决监督学习以外的问题可能需要解决可伸缩性、异构性和隐私性方面的类似挑战。

4.7 产品性联邦学习

在产品环境中运行联邦学习时还需要考虑一些实际问题。

尤其是概念漂移（当底层数据生成模型随时间变化时）；

日变化（当设备在一天或一周的不同时间表现出不同的行为时）；

冷启动问题（当新设备进入网络时）等问题必须小心处理

4.8 基准

以现实世界的环境、假设和数据集为基础

进一步建立在现有的实现和基准工具上，以促进经验结果的可重复性和联邦学习的新解决方案的传播。

5. TFF框架(tensorflow fedrate framwork)

• 构造数据集

  emnist_train, emnist_test = tff.simulation.datasets.emnist.load_data()
  example_dataset = emnist_train.create_tf_dataset_for_client(emnist_train.client_ids[0])
  example_element = next(iter(example_dataset))
  example_element['label'].numpy()

• 每个客户端创建数据集

  sample_clients = emnist_train.client_ids[0:NUM_CLIENTS]
  federated_train_data = make_federated_data(emnist_train, sample_clients)
  print('Number of client datasets:{l}'.format(l=len(federated_train_data)))
  print('First_dataset: {d}'.format(d = federated_train_data[0]))

• 构造 keras模型

  def create_keras_model():
      return tf.keras.models.Sequential([
          tf.keras.layers.Input(shape = (784,)),
          tf.keras.layers.Dense(10, kernel_initializer = 'zeros'),
          tf.keras.layers.Softmax(),       
      ])
  example_element = next(iter(example_dataset))
  example_element['label'].numpy()
  
  def model_fn():
      #必须在这里创建一个新模型
      keras_model = create_keras_model()
      return tff.learning.from_keras_model(
          keras_model,
          input_spec=preprocess_example_dataset.element_spec,
          loss = tf.keras.losses.SparseCategoricalCrossentropy(),
          metrics=[tf.keras.metrics.SparseCategoricalCrossentropy()]    
      )

• 构造迭代器（每一轮要做什么）

  iterative_process = tff.learning.build_federated_averaging_process(
  model_fn,
  client_optimizer_fn= lambda: tf.keras.optimizers.SGD(learning_rate=0.02),
  server_optimizer_fn= lambda: tf.keras.optimizers.SGD(learning_rate=1))

• 训练

  NUM_ROUNDS = 11
  trainer = tff.learning.build_federated_averaging_process(
              model_fn,
              client_optimizer_fn=lambda:tf.keras.optimizers.SGD(0.1))
  
  state = trainer.initialize()
  for round_num in range(2,NUM_ROUNDS):
      state, metrics = iterative_process.next(state, federated_train_data)
      print('round{:2d}, metrics = {}'.format(round_num, metrics))

6. pytorch 版实现（暂时没有框架，都是自己手写)

6.1 fedAvg

# 本地训练就是 batch-sgd    
for epoch in tqdm(range(self.local_epoch)):
        train_loss = []
        for (seq1, label) in self.dataset:
            seq1 = seq1.to(device)
            label = label.to(device)
            y_pred = self.model(seq1)
            y_pred = y_pred.to(device)
            loss = loss_function(y_pred, label)
            train_loss.append(loss.item())
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
        lr_step.step()
    return np.mean(train_loss), self.model.named_parameters()

6.2 fedprox

for epoch in tqdm(range(self.local_epoch)):
    train_loss = []
    for (seq1, label) in self.dataset:
        seq1 = seq1.to(device)
        label = label.to(device)
        y_pred = self.model(seq1)
        y_pred = y_pred.to(device)
        loss = loss_function(y_pred, label)

        # fedprox 算法 在 fedavg 基础上加上以下代码
        # 硬造了一个超参数 proximal_term 加到loss里, 作为训练时的惩罚项, 
        # 避免本地模型参数 与 全局模型 越来越远
        if self.alogo == 'fedprox':
          proximal_term = 0.0
          for w, w_t in zip(self.model.parameters(), fedServer.model.parameters()):
 proximal_term += (w - w_t).norm(2)
              loss = loss + self.mu/2.0 * proximal_term
        # fedprox 算法 end
        train_loss.append(loss.item())
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    lr_step.step()
return np.mean(train_loss), self.model.named_parameters()

7 构造Non-IID数据集

import torch

n_clients = 100
n_class = 10
alpah = 0.1

# dataset = [1,1,1,1...1]
# m
# 使用迪利克雷函数，生成各个客户端上，所有class类别样本数的占用比例
m = torch.distributions.dirichlet.Dirichlet(torch.ones(n_class) * alpah).sample((n_clients,))
# 遍历每个客户端，打印信息
for i in range(n_clients):
    # one_client_sample_data_percent_list
    #一个客户端上，各个class类别样本数，在各个私有数据集上的占比
    one_client_sample_data_percent_list = m[i]
    total_percent = 0
    print("###################### client {} start ######################".format(i))
    for j in range(n_class):
        # one_class_percent
        # 一个类别样本数的占比
        one_class_percent = one_client_sample_data_percent_list[j]
        total_percent += one_class_percent
        print("client {} class {} , percent : {:.2%}".format(i + 1, j + 1, one_class_percent))
    print("total_p : {:.2%}".format(total_percent))
    print("###################### client {} end ######################".format(i))

结果

###################### client 0 start ######################
client 1 class 1 , percent : 0.48%
client 1 class 2 , percent : 16.92%
client 1 class 3 , percent : 11.61%
client 1 class 4 , percent : 0.01%
client 1 class 5 , percent : 0.00%
client 1 class 6 , percent : 1.87%
client 1 class 7 , percent : 0.00%
client 1 class 8 , percent : 0.00%
client 1 class 9 , percent : 69.10%
client 1 class 10 , percent : 0.01%
total_p : 100.00%
###################### client 0 end ######################
###################### client 1 start ######################
client 2 class 1 , percent : 1.89%
client 2 class 2 , percent : 0.02%
client 2 class 3 , percent : 3.48%
client 2 class 4 , percent : 1.94%
client 2 class 5 , percent : 0.00%
client 2 class 6 , percent : 4.34%
client 2 class 7 , percent : 88.33%
client 2 class 8 , percent : 0.00%
client 2 class 9 , percent : 0.00%
client 2 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 1 end ######################
###################### client 2 start ######################
client 3 class 1 , percent : 8.82%
client 3 class 2 , percent : 21.25%
client 3 class 3 , percent : 0.00%
client 3 class 4 , percent : 0.00%
client 3 class 5 , percent : 0.00%
client 3 class 6 , percent : 0.01%
client 3 class 7 , percent : 60.19%
client 3 class 8 , percent : 0.00%
client 3 class 9 , percent : 0.00%
client 3 class 10 , percent : 9.73%
total_p : 100.00%
###################### client 2 end ######################
###################### client 3 start ######################
client 4 class 1 , percent : 9.88%
client 4 class 2 , percent : 0.07%
client 4 class 3 , percent : 31.25%
client 4 class 4 , percent : 5.47%
client 4 class 5 , percent : 0.00%
client 4 class 6 , percent : 1.04%
client 4 class 7 , percent : 0.00%
client 4 class 8 , percent : 0.00%
client 4 class 9 , percent : 52.28%
client 4 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 3 end ######################
###################### client 4 start ######################
client 5 class 1 , percent : 70.58%
client 5 class 2 , percent : 0.00%
client 5 class 3 , percent : 2.25%
client 5 class 4 , percent : 0.00%
client 5 class 5 , percent : 0.02%
client 5 class 6 , percent : 0.00%
client 5 class 7 , percent : 0.00%
client 5 class 8 , percent : 0.82%
client 5 class 9 , percent : 0.00%
client 5 class 10 , percent : 26.33%
total_p : 100.00%
###################### client 4 end ######################
###################### client 5 start ######################
client 6 class 1 , percent : 0.23%
client 6 class 2 , percent : 1.94%
client 6 class 3 , percent : 47.73%
client 6 class 4 , percent : 0.35%
client 6 class 5 , percent : 7.02%
client 6 class 6 , percent : 6.82%
client 6 class 7 , percent : 0.00%
client 6 class 8 , percent : 29.83%
client 6 class 9 , percent : 6.07%
client 6 class 10 , percent : 0.01%
total_p : 100.00%
###################### client 5 end ######################
###################### client 6 start ######################
client 7 class 1 , percent : 5.09%
client 7 class 2 , percent : 34.61%
client 7 class 3 , percent : 3.02%
client 7 class 4 , percent : 0.83%
client 7 class 5 , percent : 34.86%
client 7 class 6 , percent : 0.03%
client 7 class 7 , percent : 5.56%
client 7 class 8 , percent : 0.00%
client 7 class 9 , percent : 0.48%
client 7 class 10 , percent : 15.52%
total_p : 100.00%
###################### client 6 end ######################
###################### client 7 start ######################
client 8 class 1 , percent : 7.96%
client 8 class 2 , percent : 0.00%
client 8 class 3 , percent : 0.67%
client 8 class 4 , percent : 0.00%
client 8 class 5 , percent : 0.00%
client 8 class 6 , percent : 0.00%
client 8 class 7 , percent : 6.48%
client 8 class 8 , percent : 0.04%
client 8 class 9 , percent : 0.27%
client 8 class 10 , percent : 84.57%
total_p : 100.00%
###################### client 7 end ######################
###################### client 8 start ######################
client 9 class 1 , percent : 0.21%
client 9 class 2 , percent : 0.03%
client 9 class 3 , percent : 0.34%
client 9 class 4 , percent : 62.26%
client 9 class 5 , percent : 0.00%
client 9 class 6 , percent : 0.11%
client 9 class 7 , percent : 36.86%
client 9 class 8 , percent : 0.20%
client 9 class 9 , percent : 0.00%
client 9 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 8 end ######################
###################### client 9 start ######################
client 10 class 1 , percent : 3.84%
client 10 class 2 , percent : 0.00%
client 10 class 3 , percent : 0.01%
client 10 class 4 , percent : 0.00%
client 10 class 5 , percent : 0.07%
client 10 class 6 , percent : 0.34%
client 10 class 7 , percent : 0.00%
client 10 class 8 , percent : 0.00%
client 10 class 9 , percent : 95.26%
client 10 class 10 , percent : 0.48%
total_p : 100.00%
###################### client 9 end ######################
###################### client 10 start ######################
client 11 class 1 , percent : 54.44%
client 11 class 2 , percent : 0.52%
client 11 class 3 , percent : 0.01%
client 11 class 4 , percent : 0.08%
client 11 class 5 , percent : 0.00%
client 11 class 6 , percent : 0.32%
client 11 class 7 , percent : 0.00%
client 11 class 8 , percent : 42.46%
client 11 class 9 , percent : 0.04%
client 11 class 10 , percent : 2.13%
total_p : 100.00%
###################### client 10 end ######################
###################### client 11 start ######################
client 12 class 1 , percent : 1.03%
client 12 class 2 , percent : 36.07%
client 12 class 3 , percent : 0.00%
client 12 class 4 , percent : 0.00%
client 12 class 5 , percent : 0.47%
client 12 class 6 , percent : 30.97%
client 12 class 7 , percent : 0.00%
client 12 class 8 , percent : 0.00%
client 12 class 9 , percent : 31.46%
client 12 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 11 end ######################
###################### client 12 start ######################
client 13 class 1 , percent : 23.86%
client 13 class 2 , percent : 0.00%
client 13 class 3 , percent : 0.00%
client 13 class 4 , percent : 0.00%
client 13 class 5 , percent : 0.58%
client 13 class 6 , percent : 0.10%
client 13 class 7 , percent : 75.39%
client 13 class 8 , percent : 0.00%
client 13 class 9 , percent : 0.07%
client 13 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 12 end ######################
###################### client 13 start ######################
client 14 class 1 , percent : 10.89%
client 14 class 2 , percent : 0.75%
client 14 class 3 , percent : 0.00%
client 14 class 4 , percent : 0.00%
client 14 class 5 , percent : 1.67%
client 14 class 6 , percent : 68.04%
client 14 class 7 , percent : 0.73%
client 14 class 8 , percent : 13.45%
client 14 class 9 , percent : 0.00%
client 14 class 10 , percent : 4.47%
total_p : 100.00%
###################### client 13 end ######################
###################### client 14 start ######################
client 15 class 1 , percent : 0.01%
client 15 class 2 , percent : 45.56%
client 15 class 3 , percent : 1.67%
client 15 class 4 , percent : 0.12%
client 15 class 5 , percent : 0.00%
client 15 class 6 , percent : 0.00%
client 15 class 7 , percent : 0.00%
client 15 class 8 , percent : 1.03%
client 15 class 9 , percent : 0.00%
client 15 class 10 , percent : 51.61%
total_p : 100.00%
###################### client 14 end ######################
###################### client 15 start ######################
client 16 class 1 , percent : 36.56%
client 16 class 2 , percent : 9.76%
client 16 class 3 , percent : 0.00%
client 16 class 4 , percent : 0.27%
client 16 class 5 , percent : 0.00%
client 16 class 6 , percent : 0.00%
client 16 class 7 , percent : 0.25%
client 16 class 8 , percent : 0.11%
client 16 class 9 , percent : 52.99%
client 16 class 10 , percent : 0.05%
total_p : 100.00%
###################### client 15 end ######################
###################### client 16 start ######################
client 17 class 1 , percent : 6.42%
client 17 class 2 , percent : 0.01%
client 17 class 3 , percent : 0.00%
client 17 class 4 , percent : 0.12%
client 17 class 5 , percent : 35.30%
client 17 class 6 , percent : 23.82%
client 17 class 7 , percent : 0.00%
client 17 class 8 , percent : 9.14%
client 17 class 9 , percent : 0.07%
client 17 class 10 , percent : 25.14%
total_p : 100.00%
###################### client 16 end ######################
###################### client 17 start ######################
client 18 class 1 , percent : 7.40%
client 18 class 2 , percent : 0.01%
client 18 class 3 , percent : 0.00%
client 18 class 4 , percent : 0.00%
client 18 class 5 , percent : 0.00%
client 18 class 6 , percent : 2.41%
client 18 class 7 , percent : 6.90%
client 18 class 8 , percent : 83.26%
client 18 class 9 , percent : 0.00%
client 18 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 17 end ######################
###################### client 18 start ######################
client 19 class 1 , percent : 0.49%
client 19 class 2 , percent : 0.74%
client 19 class 3 , percent : 0.00%
client 19 class 4 , percent : 93.64%
client 19 class 5 , percent : 0.43%
client 19 class 6 , percent : 2.67%
client 19 class 7 , percent : 1.54%
client 19 class 8 , percent : 0.00%
client 19 class 9 , percent : 0.16%
client 19 class 10 , percent : 0.32%
total_p : 100.00%
###################### client 18 end ######################
###################### client 19 start ######################
client 20 class 1 , percent : 0.00%
client 20 class 2 , percent : 35.66%
client 20 class 3 , percent : 0.78%
client 20 class 4 , percent : 0.00%
client 20 class 5 , percent : 7.41%
client 20 class 6 , percent : 0.52%
client 20 class 7 , percent : 51.21%
client 20 class 8 , percent : 4.02%
client 20 class 9 , percent : 0.00%
client 20 class 10 , percent : 0.39%
total_p : 100.00%
###################### client 19 end ######################
###################### client 20 start ######################
client 21 class 1 , percent : 17.98%
client 21 class 2 , percent : 30.66%
client 21 class 3 , percent : 6.41%
client 21 class 4 , percent : 4.52%
client 21 class 5 , percent : 0.00%
client 21 class 6 , percent : 6.77%
client 21 class 7 , percent : 0.26%
client 21 class 8 , percent : 0.58%
client 21 class 9 , percent : 19.47%
client 21 class 10 , percent : 13.35%
total_p : 100.00%
###################### client 20 end ######################
###################### client 21 start ######################
client 22 class 1 , percent : 5.52%
client 22 class 2 , percent : 10.26%
client 22 class 3 , percent : 0.08%
client 22 class 4 , percent : 21.47%
client 22 class 5 , percent : 3.83%
client 22 class 6 , percent : 12.03%
client 22 class 7 , percent : 0.00%
client 22 class 8 , percent : 0.77%
client 22 class 9 , percent : 0.00%
client 22 class 10 , percent : 46.03%
total_p : 100.00%
###################### client 21 end ######################
###################### client 22 start ######################
client 23 class 1 , percent : 0.34%
client 23 class 2 , percent : 0.00%
client 23 class 3 , percent : 0.00%
client 23 class 4 , percent : 0.06%
client 23 class 5 , percent : 0.00%
client 23 class 6 , percent : 92.32%
client 23 class 7 , percent : 1.90%
client 23 class 8 , percent : 0.00%
client 23 class 9 , percent : 5.38%
client 23 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 22 end ######################
###################### client 23 start ######################
client 24 class 1 , percent : 1.33%
client 24 class 2 , percent : 24.59%
client 24 class 3 , percent : 28.44%
client 24 class 4 , percent : 1.69%
client 24 class 5 , percent : 36.84%
client 24 class 6 , percent : 3.78%
client 24 class 7 , percent : 0.00%
client 24 class 8 , percent : 0.00%
client 24 class 9 , percent : 1.23%
client 24 class 10 , percent : 2.09%
total_p : 100.00%
###################### client 23 end ######################
###################### client 24 start ######################
client 25 class 1 , percent : 0.13%
client 25 class 2 , percent : 50.00%
client 25 class 3 , percent : 0.00%
client 25 class 4 , percent : 0.00%
client 25 class 5 , percent : 0.65%
client 25 class 6 , percent : 0.00%
client 25 class 7 , percent : 0.15%
client 25 class 8 , percent : 41.41%
client 25 class 9 , percent : 0.00%
client 25 class 10 , percent : 7.66%
total_p : 100.00%
###################### client 24 end ######################
###################### client 25 start ######################
client 26 class 1 , percent : 30.42%
client 26 class 2 , percent : 7.91%
client 26 class 3 , percent : 58.84%
client 26 class 4 , percent : 0.01%
client 26 class 5 , percent : 2.60%
client 26 class 6 , percent : 0.00%
client 26 class 7 , percent : 0.03%
client 26 class 8 , percent : 0.00%
client 26 class 9 , percent : 0.18%
client 26 class 10 , percent : 0.02%
total_p : 100.00%
###################### client 25 end ######################
###################### client 26 start ######################
client 27 class 1 , percent : 4.68%
client 27 class 2 , percent : 59.85%
client 27 class 3 , percent : 0.07%
client 27 class 4 , percent : 0.00%
client 27 class 5 , percent : 4.99%
client 27 class 6 , percent : 30.25%
client 27 class 7 , percent : 0.00%
client 27 class 8 , percent : 0.01%
client 27 class 9 , percent : 0.02%
client 27 class 10 , percent : 0.12%
total_p : 100.00%
###################### client 26 end ######################
###################### client 27 start ######################
client 28 class 1 , percent : 0.00%
client 28 class 2 , percent : 0.01%
client 28 class 3 , percent : 6.79%
client 28 class 4 , percent : 0.01%
client 28 class 5 , percent : 0.22%
client 28 class 6 , percent : 0.00%
client 28 class 7 , percent : 92.12%
client 28 class 8 , percent : 0.85%
client 28 class 9 , percent : 0.00%
client 28 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 27 end ######################
###################### client 28 start ######################
client 29 class 1 , percent : 34.53%
client 29 class 2 , percent : 0.00%
client 29 class 3 , percent : 0.04%
client 29 class 4 , percent : 0.00%
client 29 class 5 , percent : 0.00%
client 29 class 6 , percent : 6.76%
client 29 class 7 , percent : 17.56%
client 29 class 8 , percent : 30.81%
client 29 class 9 , percent : 8.78%
client 29 class 10 , percent : 1.51%
total_p : 100.00%
###################### client 28 end ######################
###################### client 29 start ######################
client 30 class 1 , percent : 0.66%
client 30 class 2 , percent : 0.85%
client 30 class 3 , percent : 0.00%
client 30 class 4 , percent : 55.45%
client 30 class 5 , percent : 0.03%
client 30 class 6 , percent : 37.72%
client 30 class 7 , percent : 1.74%
client 30 class 8 , percent : 3.41%
client 30 class 9 , percent : 0.00%
client 30 class 10 , percent : 0.15%
total_p : 100.00%
###################### client 29 end ######################
###################### client 30 start ######################
client 31 class 1 , percent : 72.03%
client 31 class 2 , percent : 0.00%
client 31 class 3 , percent : 0.01%
client 31 class 4 , percent : 0.00%
client 31 class 5 , percent : 1.54%
client 31 class 6 , percent : 0.01%
client 31 class 7 , percent : 0.20%
client 31 class 8 , percent : 25.92%
client 31 class 9 , percent : 0.29%
client 31 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 30 end ######################
###################### client 31 start ######################
client 32 class 1 , percent : 1.27%
client 32 class 2 , percent : 0.00%
client 32 class 3 , percent : 0.00%
client 32 class 4 , percent : 0.00%
client 32 class 5 , percent : 0.00%
client 32 class 6 , percent : 15.00%
client 32 class 7 , percent : 0.00%
client 32 class 8 , percent : 0.05%
client 32 class 9 , percent : 83.10%
client 32 class 10 , percent : 0.57%
total_p : 100.00%
###################### client 31 end ######################
###################### client 32 start ######################
client 33 class 1 , percent : 38.01%
client 33 class 2 , percent : 0.00%
client 33 class 3 , percent : 0.00%
client 33 class 4 , percent : 0.00%
client 33 class 5 , percent : 0.00%
client 33 class 6 , percent : 30.01%
client 33 class 7 , percent : 31.34%
client 33 class 8 , percent : 0.58%
client 33 class 9 , percent : 0.00%
client 33 class 10 , percent : 0.05%
total_p : 100.00%
###################### client 32 end ######################
###################### client 33 start ######################
client 34 class 1 , percent : 5.36%
client 34 class 2 , percent : 1.86%
client 34 class 3 , percent : 59.90%
client 34 class 4 , percent : 1.32%
client 34 class 5 , percent : 5.21%
client 34 class 6 , percent : 0.04%
client 34 class 7 , percent : 0.00%
client 34 class 8 , percent : 0.04%
client 34 class 9 , percent : 26.26%
client 34 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 33 end ######################
###################### client 34 start ######################
client 35 class 1 , percent : 0.04%
client 35 class 2 , percent : 2.13%
client 35 class 3 , percent : 70.56%
client 35 class 4 , percent : 0.00%
client 35 class 5 , percent : 14.04%
client 35 class 6 , percent : 0.00%
client 35 class 7 , percent : 1.43%
client 35 class 8 , percent : 1.23%
client 35 class 9 , percent : 10.30%
client 35 class 10 , percent : 0.27%
total_p : 100.00%
###################### client 34 end ######################
###################### client 35 start ######################
client 36 class 1 , percent : 0.00%
client 36 class 2 , percent : 0.00%
client 36 class 3 , percent : 0.00%
client 36 class 4 , percent : 0.00%
client 36 class 5 , percent : 0.00%
client 36 class 6 , percent : 1.32%
client 36 class 7 , percent : 0.00%
client 36 class 8 , percent : 0.00%
client 36 class 9 , percent : 0.00%
client 36 class 10 , percent : 98.67%
total_p : 100.00%
###################### client 35 end ######################
###################### client 36 start ######################
client 37 class 1 , percent : 0.00%
client 37 class 2 , percent : 0.00%
client 37 class 3 , percent : 0.03%
client 37 class 4 , percent : 4.23%
client 37 class 5 , percent : 2.36%
client 37 class 6 , percent : 0.00%
client 37 class 7 , percent : 5.61%
client 37 class 8 , percent : 0.00%
client 37 class 9 , percent : 0.01%
client 37 class 10 , percent : 87.75%
total_p : 100.00%
###################### client 36 end ######################
###################### client 37 start ######################
client 38 class 1 , percent : 0.06%
client 38 class 2 , percent : 0.02%
client 38 class 3 , percent : 3.17%
client 38 class 4 , percent : 0.00%
client 38 class 5 , percent : 0.01%
client 38 class 6 , percent : 0.00%
client 38 class 7 , percent : 0.00%
client 38 class 8 , percent : 0.00%
client 38 class 9 , percent : 0.06%
client 38 class 10 , percent : 96.67%
total_p : 100.00%
###################### client 37 end ######################
###################### client 38 start ######################
client 39 class 1 , percent : 0.01%
client 39 class 2 , percent : 2.34%
client 39 class 3 , percent : 74.67%
client 39 class 4 , percent : 0.00%
client 39 class 5 , percent : 0.58%
client 39 class 6 , percent : 0.00%
client 39 class 7 , percent : 0.01%
client 39 class 8 , percent : 1.56%
client 39 class 9 , percent : 0.00%
client 39 class 10 , percent : 20.82%
total_p : 100.00%
###################### client 38 end ######################
###################### client 39 start ######################
client 40 class 1 , percent : 9.24%
client 40 class 2 , percent : 2.07%
client 40 class 3 , percent : 0.00%
client 40 class 4 , percent : 6.50%
client 40 class 5 , percent : 0.02%
client 40 class 6 , percent : 82.18%
client 40 class 7 , percent : 0.00%
client 40 class 8 , percent : 0.00%
client 40 class 9 , percent : 0.00%
client 40 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 39 end ######################
###################### client 40 start ######################
client 41 class 1 , percent : 7.17%
client 41 class 2 , percent : 9.65%
client 41 class 3 , percent : 2.00%
client 41 class 4 , percent : 0.00%
client 41 class 5 , percent : 0.00%
client 41 class 6 , percent : 0.00%
client 41 class 7 , percent : 0.00%
client 41 class 8 , percent : 0.00%
client 41 class 9 , percent : 57.97%
client 41 class 10 , percent : 23.21%
total_p : 100.00%
###################### client 40 end ######################
###################### client 41 start ######################
client 42 class 1 , percent : 0.00%
client 42 class 2 , percent : 3.99%
client 42 class 3 , percent : 3.10%
client 42 class 4 , percent : 0.00%
client 42 class 5 , percent : 92.07%
client 42 class 6 , percent : 0.30%
client 42 class 7 , percent : 0.12%
client 42 class 8 , percent : 0.01%
client 42 class 9 , percent : 0.39%
client 42 class 10 , percent : 0.01%
total_p : 100.00%
###################### client 41 end ######################
###################### client 42 start ######################
client 43 class 1 , percent : 0.00%
client 43 class 2 , percent : 75.90%
client 43 class 3 , percent : 1.48%
client 43 class 4 , percent : 0.00%
client 43 class 5 , percent : 0.58%
client 43 class 6 , percent : 0.00%
client 43 class 7 , percent : 0.00%
client 43 class 8 , percent : 0.14%
client 43 class 9 , percent : 21.90%
client 43 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 42 end ######################
###################### client 43 start ######################
client 44 class 1 , percent : 0.00%
client 44 class 2 , percent : 3.06%
client 44 class 3 , percent : 0.00%
client 44 class 4 , percent : 0.01%
client 44 class 5 , percent : 0.00%
client 44 class 6 , percent : 13.88%
client 44 class 7 , percent : 0.03%
client 44 class 8 , percent : 0.70%
client 44 class 9 , percent : 82.32%
client 44 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 43 end ######################
###################### client 44 start ######################
client 45 class 1 , percent : 0.00%
client 45 class 2 , percent : 0.00%
client 45 class 3 , percent : 0.14%
client 45 class 4 , percent : 0.00%
client 45 class 5 , percent : 0.00%
client 45 class 6 , percent : 0.00%
client 45 class 7 , percent : 0.02%
client 45 class 8 , percent : 0.00%
client 45 class 9 , percent : 0.00%
client 45 class 10 , percent : 99.85%
total_p : 100.00%
###################### client 44 end ######################
###################### client 45 start ######################
client 46 class 1 , percent : 0.24%
client 46 class 2 , percent : 0.01%
client 46 class 3 , percent : 0.00%
client 46 class 4 , percent : 43.92%
client 46 class 5 , percent : 0.50%
client 46 class 6 , percent : 1.25%
client 46 class 7 , percent : 50.91%
client 46 class 8 , percent : 0.00%
client 46 class 9 , percent : 1.16%
client 46 class 10 , percent : 2.01%
total_p : 100.00%
###################### client 45 end ######################
###################### client 46 start ######################
client 47 class 1 , percent : 0.00%
client 47 class 2 , percent : 0.00%
client 47 class 3 , percent : 0.00%
client 47 class 4 , percent : 0.00%
client 47 class 5 , percent : 99.48%
client 47 class 6 , percent : 0.06%
client 47 class 7 , percent : 0.46%
client 47 class 8 , percent : 0.00%
client 47 class 9 , percent : 0.00%
client 47 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 46 end ######################
###################### client 47 start ######################
client 48 class 1 , percent : 3.26%
client 48 class 2 , percent : 0.13%
client 48 class 3 , percent : 0.00%
client 48 class 4 , percent : 22.51%
client 48 class 5 , percent : 0.00%
client 48 class 6 , percent : 0.00%
client 48 class 7 , percent : 65.13%
client 48 class 8 , percent : 0.31%
client 48 class 9 , percent : 0.31%
client 48 class 10 , percent : 8.35%
total_p : 100.00%
###################### client 47 end ######################
###################### client 48 start ######################
client 49 class 1 , percent : 0.00%
client 49 class 2 , percent : 58.83%
client 49 class 3 , percent : 24.92%
client 49 class 4 , percent : 0.00%
client 49 class 5 , percent : 0.02%
client 49 class 6 , percent : 15.73%
client 49 class 7 , percent : 0.00%
client 49 class 8 , percent : 0.41%
client 49 class 9 , percent : 0.08%
client 49 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 48 end ######################
###################### client 49 start ######################
client 50 class 1 , percent : 0.00%
client 50 class 2 , percent : 18.50%
client 50 class 3 , percent : 0.00%
client 50 class 4 , percent : 0.00%
client 50 class 5 , percent : 74.98%
client 50 class 6 , percent : 1.53%
client 50 class 7 , percent : 4.99%
client 50 class 8 , percent : 0.00%
client 50 class 9 , percent : 0.00%
client 50 class 10 , percent : 0.01%
total_p : 100.00%
###################### client 49 end ######################
###################### client 50 start ######################
client 51 class 1 , percent : 0.06%
client 51 class 2 , percent : 0.00%
client 51 class 3 , percent : 10.89%
client 51 class 4 , percent : 0.00%
client 51 class 5 , percent : 0.00%
client 51 class 6 , percent : 0.24%
client 51 class 7 , percent : 10.03%
client 51 class 8 , percent : 78.70%
client 51 class 9 , percent : 0.00%
client 51 class 10 , percent : 0.08%
total_p : 100.00%
###################### client 50 end ######################
###################### client 51 start ######################
client 52 class 1 , percent : 36.48%
client 52 class 2 , percent : 0.00%
client 52 class 3 , percent : 61.96%
client 52 class 4 , percent : 0.00%
client 52 class 5 , percent : 0.01%
client 52 class 6 , percent : 0.00%
client 52 class 7 , percent : 0.00%
client 52 class 8 , percent : 1.31%
client 52 class 9 , percent : 0.21%
client 52 class 10 , percent : 0.03%
total_p : 100.00%
###################### client 51 end ######################
###################### client 52 start ######################
client 53 class 1 , percent : 0.00%
client 53 class 2 , percent : 0.72%
client 53 class 3 , percent : 0.02%
client 53 class 4 , percent : 0.00%
client 53 class 5 , percent : 0.00%
client 53 class 6 , percent : 91.12%
client 53 class 7 , percent : 0.00%
client 53 class 8 , percent : 1.30%
client 53 class 9 , percent : 0.07%
client 53 class 10 , percent : 6.77%
total_p : 100.00%
###################### client 52 end ######################
###################### client 53 start ######################
client 54 class 1 , percent : 67.31%
client 54 class 2 , percent : 0.01%
client 54 class 3 , percent : 0.25%
client 54 class 4 , percent : 28.29%
client 54 class 5 , percent : 0.81%
client 54 class 6 , percent : 3.27%
client 54 class 7 , percent : 0.04%
client 54 class 8 , percent : 0.00%
client 54 class 9 , percent : 0.00%
client 54 class 10 , percent : 0.01%
total_p : 100.00%
###################### client 53 end ######################
###################### client 54 start ######################
client 55 class 1 , percent : 0.00%
client 55 class 2 , percent : 0.00%
client 55 class 3 , percent : 0.08%
client 55 class 4 , percent : 0.00%
client 55 class 5 , percent : 99.48%
client 55 class 6 , percent : 0.01%
client 55 class 7 , percent : 0.24%
client 55 class 8 , percent : 0.00%
client 55 class 9 , percent : 0.00%
client 55 class 10 , percent : 0.20%
total_p : 100.00%
###################### client 54 end ######################
###################### client 55 start ######################
client 56 class 1 , percent : 0.27%
client 56 class 2 , percent : 0.00%
client 56 class 3 , percent : 29.05%
client 56 class 4 , percent : 5.18%
client 56 class 5 , percent : 0.00%
client 56 class 6 , percent : 7.08%
client 56 class 7 , percent : 0.08%
client 56 class 8 , percent : 7.77%
client 56 class 9 , percent : 0.00%
client 56 class 10 , percent : 50.56%
total_p : 100.00%
###################### client 55 end ######################
###################### client 56 start ######################
client 57 class 1 , percent : 0.10%
client 57 class 2 , percent : 0.00%
client 57 class 3 , percent : 0.01%
client 57 class 4 , percent : 0.00%
client 57 class 5 , percent : 0.00%
client 57 class 6 , percent : 0.00%
client 57 class 7 , percent : 0.00%
client 57 class 8 , percent : 29.20%
client 57 class 9 , percent : 70.69%
client 57 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 56 end ######################
###################### client 57 start ######################
client 58 class 1 , percent : 0.00%
client 58 class 2 , percent : 0.27%
client 58 class 3 , percent : 0.00%
client 58 class 4 , percent : 2.05%
client 58 class 5 , percent : 9.61%
client 58 class 6 , percent : 0.20%
client 58 class 7 , percent : 0.00%
client 58 class 8 , percent : 0.00%
client 58 class 9 , percent : 0.00%
client 58 class 10 , percent : 87.87%
total_p : 100.00%
###################### client 57 end ######################
###################### client 58 start ######################
client 59 class 1 , percent : 0.08%
client 59 class 2 , percent : 0.01%
client 59 class 3 , percent : 0.00%
client 59 class 4 , percent : 2.40%
client 59 class 5 , percent : 95.63%
client 59 class 6 , percent : 0.00%
client 59 class 7 , percent : 0.35%
client 59 class 8 , percent : 0.21%
client 59 class 9 , percent : 0.00%
client 59 class 10 , percent : 1.32%
total_p : 100.00%
###################### client 58 end ######################
###################### client 59 start ######################
client 60 class 1 , percent : 0.00%
client 60 class 2 , percent : 2.60%
client 60 class 3 , percent : 0.01%
client 60 class 4 , percent : 1.89%
client 60 class 5 , percent : 54.03%
client 60 class 6 , percent : 0.00%
client 60 class 7 , percent : 10.57%
client 60 class 8 , percent : 0.00%
client 60 class 9 , percent : 0.00%
client 60 class 10 , percent : 30.91%
total_p : 100.00%
###################### client 59 end ######################
###################### client 60 start ######################
client 61 class 1 , percent : 0.30%
client 61 class 2 , percent : 5.54%
client 61 class 3 , percent : 0.00%
client 61 class 4 , percent : 0.02%
client 61 class 5 , percent : 38.25%
client 61 class 6 , percent : 0.00%
client 61 class 7 , percent : 38.65%
client 61 class 8 , percent : 0.00%
client 61 class 9 , percent : 17.18%
client 61 class 10 , percent : 0.05%
total_p : 100.00%
###################### client 60 end ######################
###################### client 61 start ######################
client 62 class 1 , percent : 0.00%
client 62 class 2 , percent : 0.05%
client 62 class 3 , percent : 2.17%
client 62 class 4 , percent : 2.41%
client 62 class 5 , percent : 90.66%
client 62 class 6 , percent : 0.00%
client 62 class 7 , percent : 4.71%
client 62 class 8 , percent : 0.00%
client 62 class 9 , percent : 0.00%
client 62 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 61 end ######################
###################### client 62 start ######################
client 63 class 1 , percent : 0.00%
client 63 class 2 , percent : 0.01%
client 63 class 3 , percent : 0.00%
client 63 class 4 , percent : 0.42%
client 63 class 5 , percent : 0.00%
client 63 class 6 , percent : 92.77%
client 63 class 7 , percent : 0.00%
client 63 class 8 , percent : 0.00%
client 63 class 9 , percent : 6.21%
client 63 class 10 , percent : 0.59%
total_p : 100.00%
###################### client 62 end ######################
###################### client 63 start ######################
client 64 class 1 , percent : 0.08%
client 64 class 2 , percent : 2.61%
client 64 class 3 , percent : 41.92%
client 64 class 4 , percent : 0.01%
client 64 class 5 , percent : 55.07%
client 64 class 6 , percent : 0.00%
client 64 class 7 , percent : 0.00%
client 64 class 8 , percent : 0.00%
client 64 class 9 , percent : 0.32%
client 64 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 63 end ######################
###################### client 64 start ######################
client 65 class 1 , percent : 0.00%
client 65 class 2 , percent : 0.18%
client 65 class 3 , percent : 8.37%
client 65 class 4 , percent : 1.18%
client 65 class 5 , percent : 4.81%
client 65 class 6 , percent : 17.55%
client 65 class 7 , percent : 56.99%
client 65 class 8 , percent : 1.97%
client 65 class 9 , percent : 8.92%
client 65 class 10 , percent : 0.04%
total_p : 100.00%
###################### client 64 end ######################
###################### client 65 start ######################
client 66 class 1 , percent : 0.00%
client 66 class 2 , percent : 0.00%
client 66 class 3 , percent : 0.00%
client 66 class 4 , percent : 0.00%
client 66 class 5 , percent : 0.00%
client 66 class 6 , percent : 0.00%
client 66 class 7 , percent : 77.55%
client 66 class 8 , percent : 0.00%
client 66 class 9 , percent : 22.43%
client 66 class 10 , percent : 0.02%
total_p : 100.00%
###################### client 65 end ######################
###################### client 66 start ######################
client 67 class 1 , percent : 0.00%
client 67 class 2 , percent : 0.00%
client 67 class 3 , percent : 0.00%
client 67 class 4 , percent : 0.00%
client 67 class 5 , percent : 0.00%
client 67 class 6 , percent : 99.12%
client 67 class 7 , percent : 0.05%
client 67 class 8 , percent : 0.63%
client 67 class 9 , percent : 0.19%
client 67 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 66 end ######################
###################### client 67 start ######################
client 68 class 1 , percent : 0.42%
client 68 class 2 , percent : 1.56%
client 68 class 3 , percent : 0.00%
client 68 class 4 , percent : 97.52%
client 68 class 5 , percent : 0.00%
client 68 class 6 , percent : 0.46%
client 68 class 7 , percent : 0.00%
client 68 class 8 , percent : 0.00%
client 68 class 9 , percent : 0.00%
client 68 class 10 , percent : 0.05%
total_p : 100.00%
###################### client 67 end ######################
###################### client 68 start ######################
client 69 class 1 , percent : 0.00%
client 69 class 2 , percent : 0.02%
client 69 class 3 , percent : 0.00%
client 69 class 4 , percent : 7.01%
client 69 class 5 , percent : 0.02%
client 69 class 6 , percent : 0.01%
client 69 class 7 , percent : 0.00%
client 69 class 8 , percent : 0.00%
client 69 class 9 , percent : 0.23%
client 69 class 10 , percent : 92.71%
total_p : 100.00%
###################### client 68 end ######################
###################### client 69 start ######################
client 70 class 1 , percent : 0.00%
client 70 class 2 , percent : 1.53%
client 70 class 3 , percent : 0.00%
client 70 class 4 , percent : 0.00%
client 70 class 5 , percent : 0.03%
client 70 class 6 , percent : 0.00%
client 70 class 7 , percent : 18.49%
client 70 class 8 , percent : 0.00%
client 70 class 9 , percent : 0.00%
client 70 class 10 , percent : 79.94%
total_p : 100.00%
###################### client 69 end ######################
###################### client 70 start ######################
client 71 class 1 , percent : 0.00%
client 71 class 2 , percent : 0.00%
client 71 class 3 , percent : 0.27%
client 71 class 4 , percent : 0.01%
client 71 class 5 , percent : 0.00%
client 71 class 6 , percent : 4.68%
client 71 class 7 , percent : 94.83%
client 71 class 8 , percent : 0.00%
client 71 class 9 , percent : 0.20%
client 71 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 70 end ######################
###################### client 71 start ######################
client 72 class 1 , percent : 0.40%
client 72 class 2 , percent : 0.16%
client 72 class 3 , percent : 41.70%
client 72 class 4 , percent : 0.09%
client 72 class 5 , percent : 30.37%
client 72 class 6 , percent : 3.34%
client 72 class 7 , percent : 8.07%
client 72 class 8 , percent : 0.00%
client 72 class 9 , percent : 0.08%
client 72 class 10 , percent : 15.78%
total_p : 100.00%
###################### client 71 end ######################
###################### client 72 start ######################
client 73 class 1 , percent : 0.00%
client 73 class 2 , percent : 90.53%
client 73 class 3 , percent : 0.01%
client 73 class 4 , percent : 0.32%
client 73 class 5 , percent : 0.00%
client 73 class 6 , percent : 0.00%
client 73 class 7 , percent : 5.24%
client 73 class 8 , percent : 0.62%
client 73 class 9 , percent : 3.06%
client 73 class 10 , percent : 0.22%
total_p : 100.00%
###################### client 72 end ######################
###################### client 73 start ######################
client 74 class 1 , percent : 1.58%
client 74 class 2 , percent : 0.00%
client 74 class 3 , percent : 2.53%
client 74 class 4 , percent : 0.78%
client 74 class 5 , percent : 0.68%
client 74 class 6 , percent : 0.00%
client 74 class 7 , percent : 93.94%
client 74 class 8 , percent : 0.00%
client 74 class 9 , percent : 0.00%
client 74 class 10 , percent : 0.48%
total_p : 100.00%
###################### client 73 end ######################
###################### client 74 start ######################
client 75 class 1 , percent : 0.00%
client 75 class 2 , percent : 30.59%
client 75 class 3 , percent : 40.39%
client 75 class 4 , percent : 0.00%
client 75 class 5 , percent : 0.00%
client 75 class 6 , percent : 0.54%
client 75 class 7 , percent : 0.00%
client 75 class 8 , percent : 1.08%
client 75 class 9 , percent : 27.26%
client 75 class 10 , percent : 0.14%
total_p : 100.00%
###################### client 74 end ######################
###################### client 75 start ######################
client 76 class 1 , percent : 0.26%
client 76 class 2 , percent : 17.07%
client 76 class 3 , percent : 0.64%
client 76 class 4 , percent : 0.00%
client 76 class 5 , percent : 0.06%
client 76 class 6 , percent : 0.00%
client 76 class 7 , percent : 0.25%
client 76 class 8 , percent : 81.69%
client 76 class 9 , percent : 0.00%
client 76 class 10 , percent : 0.03%
total_p : 100.00%
###################### client 75 end ######################
###################### client 76 start ######################
client 77 class 1 , percent : 92.17%
client 77 class 2 , percent : 0.00%
client 77 class 3 , percent : 0.00%
client 77 class 4 , percent : 4.49%
client 77 class 5 , percent : 0.03%
client 77 class 6 , percent : 0.00%
client 77 class 7 , percent : 0.00%
client 77 class 8 , percent : 0.00%
client 77 class 9 , percent : 0.42%
client 77 class 10 , percent : 2.89%
total_p : 100.00%
###################### client 76 end ######################
###################### client 77 start ######################
client 78 class 1 , percent : 78.95%
client 78 class 2 , percent : 2.10%
client 78 class 3 , percent : 7.73%
client 78 class 4 , percent : 0.00%
client 78 class 5 , percent : 9.43%
client 78 class 6 , percent : 1.75%
client 78 class 7 , percent : 0.04%
client 78 class 8 , percent : 0.00%
client 78 class 9 , percent : 0.00%
client 78 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 77 end ######################
###################### client 78 start ######################
client 79 class 1 , percent : 2.01%
client 79 class 2 , percent : 91.05%
client 79 class 3 , percent : 0.00%
client 79 class 4 , percent : 0.00%
client 79 class 5 , percent : 3.23%
client 79 class 6 , percent : 1.58%
client 79 class 7 , percent : 0.00%
client 79 class 8 , percent : 0.00%
client 79 class 9 , percent : 2.12%
client 79 class 10 , percent : 0.01%
total_p : 100.00%
###################### client 78 end ######################
###################### client 79 start ######################
client 80 class 1 , percent : 3.58%
client 80 class 2 , percent : 0.00%
client 80 class 3 , percent : 7.42%
client 80 class 4 , percent : 1.40%
client 80 class 5 , percent : 35.15%
client 80 class 6 , percent : 23.16%
client 80 class 7 , percent : 3.63%
client 80 class 8 , percent : 0.00%
client 80 class 9 , percent : 0.00%
client 80 class 10 , percent : 25.66%
total_p : 100.00%
###################### client 79 end ######################
###################### client 80 start ######################
client 81 class 1 , percent : 0.08%
client 81 class 2 , percent : 0.02%
client 81 class 3 , percent : 85.60%
client 81 class 4 , percent : 0.00%
client 81 class 5 , percent : 7.20%
client 81 class 6 , percent : 0.83%
client 81 class 7 , percent : 0.00%
client 81 class 8 , percent : 0.00%
client 81 class 9 , percent : 0.00%
client 81 class 10 , percent : 6.28%
total_p : 100.00%
###################### client 80 end ######################
###################### client 81 start ######################
client 82 class 1 , percent : 0.00%
client 82 class 2 , percent : 0.00%
client 82 class 3 , percent : 0.82%
client 82 class 4 , percent : 4.35%
client 82 class 5 , percent : 41.31%
client 82 class 6 , percent : 52.12%
client 82 class 7 , percent : 0.00%
client 82 class 8 , percent : 0.55%
client 82 class 9 , percent : 0.00%
client 82 class 10 , percent : 0.86%
total_p : 100.00%
###################### client 81 end ######################
###################### client 82 start ######################
client 83 class 1 , percent : 0.00%
client 83 class 2 , percent : 0.00%
client 83 class 3 , percent : 0.01%
client 83 class 4 , percent : 0.00%
client 83 class 5 , percent : 0.08%
client 83 class 6 , percent : 31.55%
client 83 class 7 , percent : 0.16%
client 83 class 8 , percent : 65.56%
client 83 class 9 , percent : 2.48%
client 83 class 10 , percent : 0.15%
total_p : 100.00%
###################### client 82 end ######################
###################### client 83 start ######################
client 84 class 1 , percent : 79.15%
client 84 class 2 , percent : 0.75%
client 84 class 3 , percent : 1.39%
client 84 class 4 , percent : 2.00%
client 84 class 5 , percent : 6.93%
client 84 class 6 , percent : 0.27%
client 84 class 7 , percent : 0.00%
client 84 class 8 , percent : 9.51%
client 84 class 9 , percent : 0.00%
client 84 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 83 end ######################
###################### client 84 start ######################
client 85 class 1 , percent : 0.00%
client 85 class 2 , percent : 0.00%
client 85 class 3 , percent : 0.00%
client 85 class 4 , percent : 0.00%
client 85 class 5 , percent : 0.00%
client 85 class 6 , percent : 92.98%
client 85 class 7 , percent : 6.73%
client 85 class 8 , percent : 0.00%
client 85 class 9 , percent : 0.20%
client 85 class 10 , percent : 0.08%
total_p : 100.00%
###################### client 84 end ######################
###################### client 85 start ######################
client 86 class 1 , percent : 0.00%
client 86 class 2 , percent : 30.61%
client 86 class 3 , percent : 0.00%
client 86 class 4 , percent : 2.67%
client 86 class 5 , percent : 62.09%
client 86 class 6 , percent : 0.00%
client 86 class 7 , percent : 0.00%
client 86 class 8 , percent : 4.52%
client 86 class 9 , percent : 0.11%
client 86 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 85 end ######################
###################### client 86 start ######################
client 87 class 1 , percent : 0.00%
client 87 class 2 , percent : 0.00%
client 87 class 3 , percent : 19.09%
client 87 class 4 , percent : 0.00%
client 87 class 5 , percent : 0.02%
client 87 class 6 , percent : 3.02%
client 87 class 7 , percent : 0.00%
client 87 class 8 , percent : 1.66%
client 87 class 9 , percent : 76.21%
client 87 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 86 end ######################
###################### client 87 start ######################
client 88 class 1 , percent : 20.86%
client 88 class 2 , percent : 0.00%
client 88 class 3 , percent : 0.00%
client 88 class 4 , percent : 52.60%
client 88 class 5 , percent : 20.40%
client 88 class 6 , percent : 0.02%
client 88 class 7 , percent : 2.68%
client 88 class 8 , percent : 0.00%
client 88 class 9 , percent : 3.41%
client 88 class 10 , percent : 0.02%
total_p : 100.00%
###################### client 87 end ######################
###################### client 88 start ######################
client 89 class 1 , percent : 0.00%
client 89 class 2 , percent : 96.17%
client 89 class 3 , percent : 3.48%
client 89 class 4 , percent : 0.03%
client 89 class 5 , percent : 0.00%
client 89 class 6 , percent : 0.00%
client 89 class 7 , percent : 0.00%
client 89 class 8 , percent : 0.31%
client 89 class 9 , percent : 0.00%
client 89 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 88 end ######################
###################### client 89 start ######################
client 90 class 1 , percent : 54.56%
client 90 class 2 , percent : 6.51%
client 90 class 3 , percent : 0.00%
client 90 class 4 , percent : 1.80%
client 90 class 5 , percent : 0.01%
client 90 class 6 , percent : 14.34%
client 90 class 7 , percent : 2.75%
client 90 class 8 , percent : 0.00%
client 90 class 9 , percent : 0.00%
client 90 class 10 , percent : 20.03%
total_p : 100.00%
###################### client 89 end ######################
###################### client 90 start ######################
client 91 class 1 , percent : 0.90%
client 91 class 2 , percent : 52.04%
client 91 class 3 , percent : 0.00%
client 91 class 4 , percent : 16.26%
client 91 class 5 , percent : 0.04%
client 91 class 6 , percent : 0.00%
client 91 class 7 , percent : 2.54%
client 91 class 8 , percent : 27.63%
client 91 class 9 , percent : 0.56%
client 91 class 10 , percent : 0.03%
total_p : 100.00%
###################### client 90 end ######################
###################### client 91 start ######################
client 92 class 1 , percent : 64.59%
client 92 class 2 , percent : 31.37%
client 92 class 3 , percent : 0.23%
client 92 class 4 , percent : 0.00%
client 92 class 5 , percent : 0.00%
client 92 class 6 , percent : 0.00%
client 92 class 7 , percent : 0.01%
client 92 class 8 , percent : 0.00%
client 92 class 9 , percent : 3.80%
client 92 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 91 end ######################
###################### client 92 start ######################
client 93 class 1 , percent : 0.00%
client 93 class 2 , percent : 20.79%
client 93 class 3 , percent : 22.45%
client 93 class 4 , percent : 20.79%
client 93 class 5 , percent : 0.00%
client 93 class 6 , percent : 0.02%
client 93 class 7 , percent : 0.00%
client 93 class 8 , percent : 0.00%
client 93 class 9 , percent : 0.02%
client 93 class 10 , percent : 35.93%
total_p : 100.00%
###################### client 92 end ######################
###################### client 93 start ######################
client 94 class 1 , percent : 0.19%
client 94 class 2 , percent : 7.93%
client 94 class 3 , percent : 0.03%
client 94 class 4 , percent : 0.00%
client 94 class 5 , percent : 0.95%
client 94 class 6 , percent : 0.00%
client 94 class 7 , percent : 0.00%
client 94 class 8 , percent : 0.00%
client 94 class 9 , percent : 81.91%
client 94 class 10 , percent : 8.99%
total_p : 100.00%
###################### client 93 end ######################
###################### client 94 start ######################
client 95 class 1 , percent : 2.52%
client 95 class 2 , percent : 0.70%
client 95 class 3 , percent : 12.84%
client 95 class 4 , percent : 0.57%
client 95 class 5 , percent : 3.93%
client 95 class 6 , percent : 79.42%
client 95 class 7 , percent : 0.00%
client 95 class 8 , percent : 0.03%
client 95 class 9 , percent : 0.00%
client 95 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 94 end ######################
###################### client 95 start ######################
client 96 class 1 , percent : 0.00%
client 96 class 2 , percent : 0.00%
client 96 class 3 , percent : 3.41%
client 96 class 4 , percent : 74.52%
client 96 class 5 , percent : 0.01%
client 96 class 6 , percent : 7.14%
client 96 class 7 , percent : 0.01%
client 96 class 8 , percent : 0.00%
client 96 class 9 , percent : 12.17%
client 96 class 10 , percent : 2.74%
total_p : 100.00%
###################### client 95 end ######################
###################### client 96 start ######################
client 97 class 1 , percent : 17.25%
client 97 class 2 , percent : 0.47%
client 97 class 3 , percent : 29.02%
client 97 class 4 , percent : 0.01%
client 97 class 5 , percent : 0.01%
client 97 class 6 , percent : 0.00%
client 97 class 7 , percent : 33.41%
client 97 class 8 , percent : 2.04%
client 97 class 9 , percent : 17.80%
client 97 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 96 end ######################
###################### client 97 start ######################
client 98 class 1 , percent : 0.02%
client 98 class 2 , percent : 0.00%
client 98 class 3 , percent : 0.19%
client 98 class 4 , percent : 6.37%
client 98 class 5 , percent : 32.15%
client 98 class 6 , percent : 1.81%
client 98 class 7 , percent : 57.50%
client 98 class 8 , percent : 0.00%
client 98 class 9 , percent : 1.60%
client 98 class 10 , percent : 0.37%
total_p : 100.00%
###################### client 97 end ######################
###################### client 98 start ######################
client 99 class 1 , percent : 0.00%
client 99 class 2 , percent : 1.35%
client 99 class 3 , percent : 98.47%
client 99 class 4 , percent : 0.01%
client 99 class 5 , percent : 0.00%
client 99 class 6 , percent : 0.00%
client 99 class 7 , percent : 0.16%
client 99 class 8 , percent : 0.00%
client 99 class 9 , percent : 0.00%
client 99 class 10 , percent : 0.01%
total_p : 100.00%
###################### client 98 end ######################
###################### client 99 start ######################
client 100 class 1 , percent : 0.54%
client 100 class 2 , percent : 0.00%
client 100 class 3 , percent : 0.00%
client 100 class 4 , percent : 0.30%
client 100 class 5 , percent : 44.61%
client 100 class 6 , percent : 53.11%
client 100 class 7 , percent : 0.00%
client 100 class 8 , percent : 0.17%
client 100 class 9 , percent : 1.28%
client 100 class 10 , percent : 0.00%
total_p : 100.00%
###################### client 99 end ######################