并发运行Scala FS2流

Question

我正在尝试建模一个系统，这是一个网站的负载生成器。目前，该网站的页面都是字符串。

type Page = String
val pages: Vector[Page] = Stream.eval(Task.delay {
  new Random().alphanumeric.take(10).mkString
}).repeat.take(100).runLog.unsafeRun

该网站的用户被建模为随机开始浏览该网站:

case class User(userName: String)
def newUser: Task[User] = Task.delay {
  val random = new Random
  val userName = random.alphanumeric.take(5).mkString
  val sleepTime = random.nextInt(10000)
  println(s"${new Date}: ${Thread.currentThread.getName} Generating user ${userName} after ${sleepTime}ms")
  Thread.sleep(sleepTime)
  User(userName)
}

每个用户从一个页面过渡到另一个页面，这被建模为:

case class Transition(from: Page, to: Page, thinkTime: Int)
def newTransitions(user: User): Stream[Task, Transition] = {
  val init = Transition(pages(0), pages(1), 100)
  Stream.iterateEval(init) { t => nextTransition(user)(t) }
}
def nextTransition(user: User)(curr: Transition): Task[Transition] = Task.delay {
  val random = new Random
  val from: Page = curr.to
  val to: Page = pages(random.nextInt(pages.size))
  val thinkTime: Int = random.nextInt(10000)
  Thread.sleep(thinkTime)
  println(s"${new Date}: ${Thread.currentThread.getName} ${user} transitioning from ${from} to ${to} after thinking for ${thinkTime}ms")
  Transition(from, to, thinkTime)
}

我正在尝试运行这个模拟，对于4个用户开始随机间隔浏览这个网站，每个用户随机进入三个页面:

implicit val S = fs2.Strategy.fromFixedDaemonPool(8, threadName = "worker")
val users: Stream[Task, User] = Stream.eval(newUser).repeat
val transitions: Stream[Task, Stream[Task, Transition]] = users.map(newTransitions(_).take(3))
transitions.take(4).runLog.async.unsafeRun.map { _.runLog.async.unsafeRun }

输出如下:

Sun Oct 16 12:58:04 PDT 2016: worker-1 Generating user 97qkE after 3700ms
Sun Oct 16 12:58:08 PDT 2016: worker-1 Generating user sMIJs after 2074ms
Sun Oct 16 12:58:10 PDT 2016: worker-1 Generating user CpWWf after 1334ms
Sun Oct 16 12:58:11 PDT 2016: worker-1 Generating user xlQmj after 8825ms
Sun Oct 16 12:58:28 PDT 2016: worker-7 User(97qkE) transitioning from sLBbTbXKhO to 7ROMneDkzl after thinking for 8040ms
Sun Oct 16 12:58:34 PDT 2016: worker-7 User(97qkE) transitioning from 7ROMneDkzl to AK66SuDENi after thinking for 6227ms
Sun Oct 16 12:58:41 PDT 2016: worker-6 User(sMIJs) transitioning from sLBbTbXKhO to U2Qu8Th1oH after thinking for 6599ms
Sun Oct 16 12:58:46 PDT 2016: worker-6 User(sMIJs) transitioning from U2Qu8Th1oH to 3bRBWYUeS0 after thinking for 4920ms
Sun Oct 16 12:58:52 PDT 2016: worker-4 User(CpWWf) transitioning from sLBbTbXKhO to ZVwRydhPe9 after thinking for 6395ms
Sun Oct 16 12:58:54 PDT 2016: worker-4 User(CpWWf) transitioning from ZVwRydhPe9 to yHtly5IrMC after thinking for 1963ms
Sun Oct 16 12:58:57 PDT 2016: worker-2 User(xlQmj) transitioning from sLBbTbXKhO to 6jn7TmxW0O after thinking for 2565ms
Sun Oct 16 12:59:05 PDT 2016: worker-2 User(xlQmj) transitioning from 6jn7TmxW0O to FFROVBvpHJ after thinking for 8535ms

。没有货币，所有的随机睡眠都是按照严格的顺序发生的。

我如何改变这个程序，使这些不同的流(新用户开始浏览网站，以及每个用户所做的转换)同时发生?

---

下面是我的完整程序:

import fs2._
import scala.util.Random
import java.util.Date
object Question {
  type Page = String
  val pages: Vector[Page] = Stream.eval(Task.delay {
    new Random().alphanumeric.take(10).mkString
  }).repeat.take(100).runLog.unsafeRun
  case class User(userName: String)
  case class Transition(from: Page, to: Page, thinkTime: Int)
  def newUser: Task[User] = Task.delay {
    val random = new Random
    val userName = random.alphanumeric.take(5).mkString
    val sleepTime = random.nextInt(10000)
    println(s"${new Date}: ${Thread.currentThread.getName} Generating user ${userName} after ${sleepTime}ms")
    Thread.sleep(sleepTime)
    User(userName)
  }
  def nextTransition(user: User)(curr: Transition): Task[Transition] = Task.delay {
    val random = new Random
    val from: Page = curr.to
    val to: Page = pages(random.nextInt(pages.size))
    val thinkTime: Int = random.nextInt(10000)
    Thread.sleep(thinkTime)
    println(s"${new Date}: ${Thread.currentThread.getName} ${user} transitioning from ${from} to ${to} after thinking for ${thinkTime}ms")
    Transition(from, to, thinkTime)
  }
  def newTransitions(user: User): Stream[Task, Transition] = {
    val init = Transition(pages(0), pages(1), 100)
    Stream.iterateEval(init) { t => nextTransition(user)(t) }
  }
  def main(args: Array[String]) {
    implicit val S = fs2.Strategy.fromFixedDaemonPool(8, threadName = "worker")
    val users: Stream[Task, User] = Stream.eval(newUser).repeat
    val transitions: Stream[Task, Stream[Task, Transition]] = users.map(newTransitions(_).take(3))
    transitions.take(4).runLog.async.unsafeRun.map { _.runLog.async.unsafeRun }
  }
}

Answer 1

首先，在你的object Question上，看起来有一个错字。记住切换:

Thread.sleep(sleepTime)
println(s"${new Date}: ${Thread.currentThread.getName} Generating user ${userName} after ${sleepTime}ms")

接下来，您希望使用Task { ... }而不是Task.delay，因为您希望在工作线程上运行它。

最后，主函数的最后一行应该是这样的:

fs2.concurrent.join(maxOpen = 3)(transitions).run.unsafeRun

另外，我猜Thread.sleep只是占位符-您不会希望在实际代码中使用它们。println可以用于调试，但最好在完成后将日志记录作为单独的流构造。