PHP前端开发

如何在Python中将工作分配给一组工作线程?

百变鹏仔 1个月前 (01-20) #Python
文章标签 工作

要在一堆工作线程之间分配工作,请使用并发.futures 模块,尤其是ThreadPoolExecutor 类。

有了这个替代方案,如果您想精细控制调度算法,您可以手动编写自己的逻辑。使用队列模块创建包含作业列表的队列。 Queue 类维护一个对象列表,并具有将项目添加到队列的 .put(obj) 方法和返回项目的 .get() 方法。该类将负责必要的锁定,以确保每个作业只分发一次。

示例

以下是一个示例 -

import threading, queue, time# The worker thread gets jobs off the queue. When the queue is empty, it# assumes there will be no more work and exits.def worker():   print('Running worker')   time.sleep(0.1)   while True:      try:         arg = q.get(block=False)      except queue.Empty:         print('Worker', threading.current_thread(), end=' ')         print('queue empty')         break      else:         print('Worker', threading.current_thread(), end=' ')         print('running with argument', arg)         time.sleep(0.5)# Create a queueq = queue.Queue()# Start a pool of 5 workersfor i in range(5):   t = threading.Thread(target=worker, name='worker %i' % (i+1))   t.start()# Begin adding work to the queuefor i in range(50):   q.put(i)# Give threads time to runprint('Main thread sleeping')time.sleep(5)

输出

Running workerRunning workerRunning workerRunning workerRunning workerMain thread sleepingWorker  running with argument 0Worker  running with argument 1Worker  running with argument 2Worker  running with argument 3Worker  running with argument 4Worker  running with argument 5Worker  running with argument 6Worker  running with argument 7Worker  running with argument 8Worker  running with argument 9Worker  running with argument 10Worker  running with argument 11Worker  running with argument 12Worker  running with argument 13Worker  running with argument 14Worker  running with argument 15Worker  running with argument 16Worker  running with argument 17Worker  running with argument 18Worker  running with argument 19Worker  running with argument 20Worker  running with argument 21Worker  running with argument 22Worker  running with argument 23Worker  running with argument 24Worker  running with argument 25Worker  running with argument 26Worker  running with argument 28Worker  running with argument 29Worker  running with argument 27Worker  running with argument 30Worker  running with argument 31Worker  running with argument 32Worker  running with argument 33Worker  running with argument 34Worker  running with argument 35Worker  running with argument 36Worker  running with argument 37Worker  running with argument 38Worker  running with argument 39Worker  running with argument 40Worker  running with argument 41Worker  running with argument 42Worker  running with argument 43Worker  running with argument 44Worker  running with argument 45Worker  running with argument 46Worker  running with argument 47Worker  running with argument 48Worker  running with argument 49Worker  queue emptyWorker  queue emptyWorker  queue emptyWorker  queue emptyWorker  queue empty