We’ve been using Flowdock for internal chat and organization since early 2010. However, during the last year(s) we noticed our usage patterns gradually shifting to other tools and services, and so we recently decided to cancel our Flowdock account. Of course we needed to somehow archive the content of our flows (that’s what chat channels are called in Flowdock) before everything was going to be deleted.
As expected, Flowdock lets you export all your data, but this leaves you with large JSON files which aren’t particularly useful if you need to quickly find and re-read this one discussion from three years ago.
So we decided to write a small Ruby script that takes the messages.json file from a Flowdock export and converts it into a simple static HTML document. This file can be viewed in any browser, is easily searchable (by using Ctrl-F / Cmd-F) and even looks a bit like the original flow on Flowdock. Files and images that were posted in the flow are preserved, too (they are part of the archive you get when exporting a flow).
Usage
The script outputs HTML to stdout (and warnings, if any, to stderr). Use it like this:
1
flowdock-archive-to-html messages.json > messages.html
Including original usernames in the output
One gotcha of an exported flow is that it includes userids only, no usernames. By default, the HTML document produced by the script will therefore contain usernames like “User 123456”.
If you prefer to have the original usernames instead you need to find out the names belonging to these ids. You can use the Flowdock API to list all users. Or, if you haven’t deleted your flows on Flowdock yet, you can also search for distinctive messages in both the generated HTML document and on Flowdock to cross-reference userids and usernames. Then simply add these mappings to the @usernames hash at the beginning of the script.
Pro tip: Run the script once to get a list of all userids in your flow (it will output a warning for each userid which is not in @usernames).
The complete script
The script is also available as a gist: gist.github.com/noniq/8f40a8dccc02ac4062b3530561bdd507
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
#!/usr/bin/env ruby
# Convert an exported Flowdock flow into a static HTML document.
#
# Usage:
#
# flowdock-archive-to-html messages.json > messages.html
#
# The script assumes that there is a subdirectory `files` containing all files referenced in the exported flow. (This is exactly the
# directory structure you get if you unzip an archive downloaded from Flowdock.)
#
# Only flow events of the types “message”, “comment”, and “file” are handled and thus included in the output. To include
# other events like “user-edit” or “mail”, add the appropriate formatting code to the huge `case` statement at the end of
# this file.
require "bundler/inline"
require "erb"
require "json"
gemfile do
source "https://rubygems.org"
gem "redcarpet", "~> 2.3.0"
end
# Add “userid => username” mappings for all users you want to identify by name. Missing users will show up as “User 123456”.
@usernames = {
"0" => "Flowdock",
}
HTML_HEADER = <<~HTML
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<style type="text/css">
html {
font-family: sans-serif;
font-size: 14px;
line-height: 1.5;
}
.message {
display: flex;
flex-direction: row;
margin: 0 0 1em;
padding: 0 0 1em;
border-bottom: 1px solid #f4f4f4;
}
.message p {
margin: 0;
overflow-wrap: break-word;
}
.message pre {
background: #eee;
padding: 0.5em 1em;
max-width: 100%;
overflow: scroll;
}
.message img {
max-width: 75%;
max-height: 50vh;
}
.message blockquote {
background: #f6f6f9;
padding: 0.5em 1em;
border-left: 3px solid #9898B0;
margin: 0 0 1em;
color: #2E2E75;
}
.date {
flex: 0 14em;
color: #999;
font-size: 80%;
text-align: right;
}
.user {
flex: 0 5em;
color: #999;
font-weight: bold;
text-align: right;
margin-right: 1em;
}
.content {
min-width: 0; /* To make `max-width: 100%` work in contained elements, see http://stackoverflow.com/a/31972181/566850 */
flex: 1;
}
</style>
</head>
<body>
HTML
HTML_FOOTER = <<~HTML
</body>
</html>
HTML
@markdown = Redcarpet::Markdown.new(Redcarpet::Render::HTML.new(escape_html: true, hard_wrap: true), autolink: true, no_intra_emphasis: true, space_after_headers: true)
def h(str)
ERB::Util.h(str)
end
def markdown(str)
@markdown.render(str.to_s)
end
def render_message(message, content)
date = Time.at(message['sent'] / 1000)
user = @usernames.fetch(message['user']) do
fallback_username = "User #{message['user']}"
$stderr.puts "WARNING: No username mapping for userid #{message['user']} - using '#{fallback_username}' instead."
@usernames[message['user']] = fallback_username
end
puts <<~HTML
<div class='message'>
<div class='user'>#{h user}</div>
<div class='content'>#{content}</div>
<div class='date'>#{date.strftime('%H:%M – %b. %d, %Y')}</div>
</div>
HTML
end
puts HTML_HEADER
JSON.parse(File.read("messages.json")).each do |message|
event = message["event"]
case event
when "message", "comment"
content =
if event == "comment"
message['content']['title'].gsub(/^/, '> \1') + "\n\n" + message['content']['text']
else
message['content']
end
render_message(message, markdown(content))
when "file"
path = "files/" + message["content"]["path"].gsub(%r{\A/files/\d+/}, "").tr("/", "_")
link_text =
if message["content"].key?("image")
"<img src='#{path}'>"
else
h(message["content"]["file_name"])
end
render_message(message, "<a href='#{path}'>#{link_text}</a>")
when "action", "user-edit", "mail", "open-invitation-enable", "line", "status", "discussion", "activity"
# ignore
else
$stderr.puts "WARNING: Unknown event type in JSON data: `#{message['event']}`"
$stderr.puts message.inspect + "\n\n"
end
end
puts HTML_FOOTER
Need to transform, convert, or otherwise transmogrify data? Chances are that this can be done (better) programmatically.