cowtowncoder
Tatu Saloranta
Ok: I know (really I do) there are tons of message queues around. But would anyone be interested in an SQS replacement, open source, distr?
flangy
Adam Vandenberg
@cowtowncoder What's the "pitch" vs. using one of the tons of other systems? The "Simple" part?
@cowtowncoder What's the "pitch" vs. using one of the tons of other systems? The "Simple" part?
cowtowncoder
Tatu Saloranta
@flangy that is one, both in good and bad (no JMS, just simple queues). Proper distr, DR; open source (just wrt commercial ones).
@flangy that is one, both in good and bad (no JMS, just simple queues). Proper distr, DR; open source (just wrt commercial ones).
cowtowncoder
Tatu Saloranta
@flangy Other things include big payloads (similar to storage systems). Possibly more efficient, multi-tenant (may be shared PaaS or private
@flangy Other things include big payloads (similar to storage systems). Possibly more efficient, multi-tenant (may be shared PaaS or private
cowtowncoder
Tatu Saloranta
@flangy and as much as I hate to say it, "designed for Cloud [from ground up]", not extended from non-distr to distributed.
@flangy and as much as I hate to say it, "designed for Cloud [from ground up]", not extended from non-distr to distributed.
flangy
Adam Vandenberg
@cowtowncoder So, essentially, a proper distributed model (unlike raw JMS/tibco)
@cowtowncoder So, essentially, a proper distributed model (unlike raw JMS/tibco)
cowtowncoder
Tatu Saloranta
@flangy Yes. And as background, core would based on a distributed store I am writing at work; once it is done, most of core is reusable
@flangy Yes. And as background, core would based on a distributed store I am writing at work; once it is done, most of core is reusable
sbtourist
Sergio Bossa
@cowtowncoder Oh man, you're going to tease me. Actually, there isn't any decent, open source, distributed MQ.
@cowtowncoder Oh man, you're going to tease me. Actually, there isn't any decent, open source, distributed MQ.
cowtowncoder
Tatu Saloranta
@sbtourist this was my epiphany on the plane: what I am working on has secondary, more interesting uses. Unless I miss something obvious.
@sbtourist this was my epiphany on the plane: what I am working on has secondary, more interesting uses. Unless I miss something obvious.
skamille
Camille Fournier
@sbtourist @cowtowncoder Now I'm curious what makes a decent distributed MQ and why none of the seemingly millions of solns is any good
@sbtourist @cowtowncoder Now I'm curious what makes a decent distributed MQ and why none of the seemingly millions of solns is any good
cowtowncoder
Tatu Saloranta
@skamille @sbtourist One analogy that might help is SQL/noSQL, JMS/SQS distinction... rich, non-distributed feature set vs distributed
@skamille @sbtourist One analogy that might help is SQL/noSQL, JMS/SQS distinction... rich, non-distributed feature set vs distributed
cowtowncoder
Tatu Saloranta
@skamille @sbtourist but granted, in my case, there's also the part "what I have would make a decent MQ", which is why I am asking... :-)
@skamille @sbtourist but granted, in my case, there's also the part "what I have would make a decent MQ", which is why I am asking... :-)
skamille
Camille Fournier
@cowtowncoder @sbtourist Mmmm but don't things like kafka and hedwig already attack this problem? Lots of people seem to like kafka
@cowtowncoder @sbtourist Mmmm but don't things like kafka and hedwig already attack this problem? Lots of people seem to like kafka
cowtowncoder
Tatu Saloranta
@skamille @sbtourist There's enough room for more than just one system. As I said, lots of respect for Kafka, but its focus may be different
@skamille @sbtourist There's enough room for more than just one system. As I said, lots of respect for Kafka, but its focus may be different
stuhood
Stu Hood
@cowtowncoder @skamille @sbtourist We're planning a Hedwig deploy soonish. Replication, ordering, topic scalability, XDC support, etc.
@cowtowncoder @skamille @sbtourist We're planning a Hedwig deploy soonish. Replication, ordering, topic scalability, XDC support, etc.
strlen
Alex Feinberg
@stuhood @cowtowncoder @skamille @sbtourist Are Hedwig guarantees per topic/partition (I am only familiar in-depth with BookKeeper so far)?
@stuhood @cowtowncoder @skamille @sbtourist Are Hedwig guarantees per topic/partition (I am only familiar in-depth with BookKeeper so far)?
stuhood
Stu Hood
@strlen @cowtowncoder @skamille @sbtourist Per topic -- topics multiplexed onto ordered BK ledgers.
@strlen @cowtowncoder @skamille @sbtourist Per topic -- topics multiplexed onto ordered BK ledgers.
strlen
Alex Feinberg
@stuhood @cowtowncoder @skamille @sbtourist In K partition of topic resides on 1 box so you get TCP order guarantee for topic w/ 1 partition
@stuhood @cowtowncoder @skamille @sbtourist In K partition of topic resides on 1 box so you get TCP order guarantee for topic w/ 1 partition
johnxorz
John Corwin
@strlen @stuhood @cowtowncoder @skamille @sbtourist ordering is guaranteed per topic in a dc. Partial ordering across dcs.
@strlen @stuhood @cowtowncoder @skamille @sbtourist ordering is guaranteed per topic in a dc. Partial ordering across dcs.
strlen
Alex Feinberg
@johnxorz @stuhood @cowtowncoder @skamille @sbtourist Ah, BK striping is taken advantage of for greater throughput with per-topic ordering?
@johnxorz @stuhood @cowtowncoder @skamille @sbtourist Ah, BK striping is taken advantage of for greater throughput with per-topic ordering?
stuhood
Stu Hood
@strlen @johnxorz @cowtowncoder @skamille @sbtourist Could theoretically use the multi-region support for local striping instead. Neat idea.
@strlen @johnxorz @cowtowncoder @skamille @sbtourist Could theoretically use the multi-region support for local striping instead. Neat idea.
sbtourist
Sergio Bossa
@johnxorz Interesting, have to take a deeper look at Hedwig. /cc @strlen @stuhood @cowtowncoder @skamille
@johnxorz Interesting, have to take a deeper look at Hedwig. /cc @strlen @stuhood @cowtowncoder @skamille