Posts in GroupServer Development
I was thinking about search results when I read the following quote from Scott Prevost, in an article about the Microsoft Bling search engine. He was discussing the captions that appear in search results. “One of the challenges in developing captions is finding the right pieces of text on a page to represent that link, so semantic processing really helps. It helps pick the right sentences, sentences that may have the right concepts but not necessarily the keywords from [the user's query]. It helps us pick the piece of the sentence that's most relevant and not chop it off in places that makes it unreadable…” http://www.theregister.co.uk/2009/07/01/powerset_and_bing/ GroupServer displays captions in the search results for posts http://groupserver.org/s?t=0&p=1 Like Scott Prevost was saying, the sample caption is one that contains the search term http://groupserver.org/s?s=search&t=0&p=1 However, that is only for posts. Keywords are returned on topic searches: http://groupserver.org/s They are words that appear frequently in a topic but not frequently in other topics (the tried and true TF-IDF algorithm). Like the captions, the keywords augment the subject, providing more “information scent”. Unlike the captions, they reveal some of the deeper meaning behind the document. I was thinking that we could do a similar thing to Microsoft Bling. Calculate the keywords for a *post* and use those to select a *sentence* that characterises the document. Currently we do not have the infrastructure to do this, but Richard has plans for eventually using a full-text retrieval system to support our search system. It would be worth looking at this idea when that is integrated with GroupServer.
If you'd like to mix it up *right now* and through the week with
others interested in e-participation, send your Skype handle to:
<email obscured>
In the SUBJECT simply write: Skype
More info: http://mudball.net/pcamp09/
Multiple files can now be posted via the Web on OnlineGroups.Net. Progressive enhancement is used to add this feature to the page, so if the browser does not support JavaScript a single file should still be added. The new JavaScript code is mostly based around Multiple File Upload plugin for jQuery http://www.fyneworks.com/jquery/multiple-file-upload/ In some short user-evaluations I found that this worked better than an alternative I proposed http://groupserver.org/r/post/2S7ExyY0u5dX46i1K9JnxU The Multiple File Upload plugin adds a hidden file-input element every time a file is added, and then clears the visible input element. The visible list of files is a copy of the same data. The “X” link next to each file name is connected to a function that deletes the line from the files list, and deletes the associated input element. I tried different variants for the “X” link. Icons — such as the ones below — can look too much like file icons or bullets. Looking like a bullet is also the problem with a dash (-). I though about the words “Remove”, “Cancel, or “Delete”, but they are ambiguous. In the end I went for a bold “X” with the tool-tip “Remove this file from files list”. The trickiest part of the entire exercise was to convince Zope Formlib not to play with the data. I did this by adding some *more* JavaScript, to rename the form-fields before they were handed back to the server. This allows the From, and Message fields to be checked by Formlib, but the files can pass through without being altered. I did try and convince Formlib to accept a list of files, but it caused too many problems when it tried to render the widget for it. Some CSS work may be called for, to make the files-list spaced out correctly, but everything is functional.
The following files were added to this topic:
I have added JavaScript code to allow multiple files to be uploaded to a topic from the one file-input element. Because of the heavy use of JavaScript, I will need to do quite a bit of testing before the code goes live. I use “progressive enhancement”, so JavaScript is not required to add a single file to a topic. JavaScript is required to add multiple files, because this is not handled by the HTML 4.01 standard. When HTML 5 forms are supported the JavaScript can be dropped. http://dev.w3.org/html5/spec/Overview.html#file-upload-state One day. One glorious day.
I said that the task was common, because I disagreed with your statement “I suspect this is a common task…“. <http://groupserver.org/r/post/20QVevWCNIigWEPlauP3yh> I am sorry if I misunderstood you I am unsure what you mean by “[viewing posts outside topics] is not supported elsewhere”. Viewing posts in a purely temporal context is supported by four interfaces: *Latest* *Posts* <http://groupserver.org/groups/development/messages/posts.html> *Latest* *Posts* *Summary* <http://groupserver.org/groups/development/messages/postsSummary.html?l=60> *Search* <http://groupserver.org/s/?g=development&t=0&p=1> *Web* *Feeds* <http://groupserver.org/s/search.atom?g=development&t=0&p=1> Even viewing individual posts in a temporal order is supported (sort of): <http://groupserver.org/groups/development/messages/posts.html?start=20&end=21> The URL for each post is always changing, however, so it is not that useful. The problem with anecdotes is that they are just little stories, reliable or not. Until we have real data we can just end up in anecdote-wars ☹ People saying that they like topics comes from large surveys, where people state they use the web interface as much as email. The surveys are backed up by posting statistics that show the web interface is used for posting and viewing statistics that show the topics are viewed. Data is more useful when making design decisions for a general audience than any anecdote. You state that your use case is not weird, but then state “these same folks don't much use the web interface”. This seems a little contradictory to me: if they do not use the web interface much then surely visiting a single post is actually weird? Even weirder, how did the user come to the single post view if he or she does not use the web much? The interface emphasises topics, rightly or wrongly. In the case of groups that do not follow topics I would love to have a group that ignored topics altogether: * No topics page * Latest Posts shown by default, and * Add to Group would replace Add to Topic (I would put it at the top of the Latest Posts page). That would actually be easier¹ than figuring out which Next and Previous links we should show, and implementing them. As for long-lived topics, we have on our horizon a scheme to fold old posts in a topic out of the way, making it easier to scan recent posts. I currently get around the problem by using the Latest Post link in a topic, scrolling up a little to check other posts (and get more context), before skipping to the next topic. (I actually use the Next and Previous links that are provided in the page metadata, but I suspect I am the only person to ever use those.) [1] Creating a new group type would be easier once we have our mythical group product written. Even with our current schedule I suspect the Group Product will pip an enhancement to the lowly post-view page.
I agree that the whole question relates to an uncommon way of viewing
posts (I'm not clear why you thought I didn't). My guess (we have little
or no data) is that the most common reason for viewing a post is when
following a short link to a post. This is itself uncommon.
I also entirely agree that supporting both types of post-navigation is
not worth considering (as I said, navigating in a topic is already
supported in the topic view). My proposal is to switch to next/previous
post in the group, if for no other reason than that this is not
supported elsewhere, so we don't know whether people would use it. It's
not an uncommon way of navigating beween posts (vis email).
My use case is not weird. I know from reliable anecdotes that some
groups do not thread one bit. They just post one after the other,
regardless of the subject line. Of course, these same foks don't much
use the web interface, but if one person in the group did, they may be
glad to find that there is actually a way to follow their group's
conversations. Even when people do usually use topics well, there is the
odd break in them. Likewise, in a group with topics that span a long
time, I may only want to read one or two posts in a topic, before
finding another recent post. This provides one case of the 'related
post' feature you mention, in a simple and recognisable way.
Changing the behaviour of the First Post, Previous Post, Next Post and Last Post is an interesting one — albeit a lot of thought for a relatively minor task. The current behaviour was created to help with debugging, rather than to assist normal users, so I am open to change. I would prefer that we went with one meaning for the links, rather than providing eight similar links. Having said that, I think it would be best to stick with our current semantic for external consistency — especially as your use case is a bit weird, Dan. Maybe a related-posts link would work? A fuller explanation, for those that can be bothered, follows ☺ The current behaviour of the Previous Post and Next Post links is not based on any study of user-tasks, and I do not think that the behaviour is necessaryly correct. I originally added the links as a debugging tool, as they allow me to figure out the post that crashes a topic. (I start at the most recent post and work my way through the topic, seeing if each post renders ok.) I could still debug topics on a per-post basis, so I am not too fussed about changing the semantics of the links. I dispute that the task of viewing individual posts in a group is common, Dan. Viewing a topic and posting to a topic are common; anything done on a per-post level pales into insignificance compared to these two tasks ☺ Rightly so, as GroupServer is designed to encourage people to use topics, which is why it takes so many clicks to get to the Latest Posts page. We get a lot of feedback that the topic-centric behaviour of our groups is good, and that they are better than email (no one actually seems to use threads, outside the tech community). I would like to avoid having too many links on one page. I would also like to avoid having two links that are close together spatially and conceptually, such as the "Next Post in List" and "Next Post in Thread" that I have seen in some places. Whenever I come across these I have to think quite a lot about the link I want to click, and I always choose the Next Post in Thread link. (I hate having to think about simple things…) If we go for external consistency, then keeping the links pointing to posts in the same topic gets a boost, as Mailman works this way <http://mail.zope.org/pipermail/zope3-users/2009-May/008564.html> Google Groups does not allow you to view an individual topic, so we cannot look there for any guidance. What does Yahoo! groups do, Dan? Your example does sound weird, Dan. The person changed the topic but did not change the topic‽ While we cannot make people stick to the topic (especially using email) we do a good job of organising the mess. There is only so much we can do, sadly. One possibility is to rely on the "References" header in the email message to create a link to posts that are not in the current topic. It would be quite a lot of work to create, as we would have to extract and index all external post-identifiers (which are crated by email clients) so we can map them to topics. Alternatively we could use our keyword-generating system to suggest other topics (or posts) that are like the current one. That would also require a fair bit of work and place strain on a component that is already doing a lot of work. And for a task that is how common? Na, stick with the current scheme ☺
On the Post page, the Next and Previous post controls currently navigate within
the Topic. This task is easily supported by navigating to the topic, which is
easily done from the Post page.
Today, I wanted to find the previous post in the group, but couldn't. I suspect
this is a common task as, in many groups, topics change without reflecting a
break in the flow of conversation.
In my example of today, the author had quoted a previous post but not linked to
it.
Should the Next and Previous post navigate within the group, not the topic?
Hi Dan, <email obscured> wrote: > The client uses Javascript invoked from the group context. It communicates, not with GroupServer, however but with the chat & presence server component. This can be done, as the two will use the same domain. GroupServer would supply group membership information directly to the server. > > Both the client and server components could potentially be acquired or developed and released as independent open source components. From a quick search, I found one possible candidate for the client. > http://code.google.com/p/libraryh3lp/ > > Presumably, the archives would be handled using some combination of the features of the client, and GroupServer's indexing search system that is used for posts, files and soon to be profiles and content. > > Dan I have had a quick look at the libraryh3lp project and discovered that the only bindings that are provided are for JavaScript, which as you said covers the client side of the "independent" chat module. Currently the project is in its infancy stages.
Here's a non-technical summary of the architecture for Chat and Presence that we discussed the other day. We envisage two components, a client and a server. The server is independent from GroupServer, but supports authentication with GroupServer, possibly by integrating OAuth into GroupServer, and using that. It uses its own Postgresql database tables, and accesses those independently of GroupServer. The server will use standard protocols (eg Jabber), so that it is possible to connect to it using arbitrary clients, as long as they also comply with the standards. The client uses Javascript invoked from the group context. It communicates, not with GroupServer, however but with the chat & presence server component. This can be done, as the two will use the same domain. GroupServer would supply group membership information directly to the server. Both the client and server components could potentially be acquired or developed and released as independent open source components. From a quick search, I found one possible candidate for the client. http://code.google.com/p/libraryh3lp/ Presumably, the archives would be handled using some combination of the features of the client, and GroupServer's indexing search system that is used for posts, files and soon to be profiles and content.
Richard,
> That is very cool. I must admit I didn't know about twitterfeed when I
> made the suggestion Steve. I guess the question would be whether we
> could do any better than that.
If we did, perhaps it could be implemented along with the proposed new
chat and presence server that we discussed the other day, and that I'll
post on soon here. Both deal with polling and short messages.
On Thu, 2009-04-23 at 10:40 +1200, Dan Randow wrote:
> Once we have greater flexibility about content in the group context (the
> group home, charter, about etc pages), it will be easy to post a link to
> the Twitter feed.
That is very cool. I must admit I didn't know about twitterfeed when I
made the suggestion Steve. I guess the question would be whether we
could do any better than that.
Hi Steve, > I created this manually: > http://twitter.com/edemmpls That's cool. I can imagine some people preferring to follow forum postings this way. > What might be useful of course is a way to automate this so you could > offer Twitter along side the web feed or something or a simple place > to put in the Twitter account via the web admin. I think what you are suggesting is that we replicate twitterfeed's functionality. Why that is needed, since twitterfeed seems to work pretty well? Once we have greater flexibility about content in the group context (the group home, charter, about etc pages), it will be easy to post a link to the Twitter feed.
The other week Richard Waid and I traded notes on Twitter notification and groups. Using this: http://twitterfeed.com/ I created this manually: http://twitter.com/edemmpls I haven't quite figured out if I really set it to capture all new topics or not. Haven't promoted it yet. What might be useful of course is a way to automate this so you could offer Twitter along side the web feed or something or a simple place to put in the Twitter account via the web admin.
I have updated the GroupServer code that handles message processing. The main
change is to allow multiple files to be posted using the web. While the
back-end code is now in place, the user-interface still only allows a single
file to be added. The only user-visible change is that the "Message posted"
message, which is shown on the Topic and Start a New Topic pages after a
message has been posted, is now a link to the new message.
Once we fix the MIME-types on our development platforms I will add the
JavaScript goodness that will allow multiple files to be added using the Web.
Yes there are plans to deal to vcard attachments, but it would be better to do something smart with them, rather than just blocking them based on the mime-type. My idea from January was to parse the vcard, probably using the "vobject" library http://pypi.python.org/pypi/vobject/ and only discard the attachment if the information is for the same person that posted the message. Otherwise you could not post the vcard for anyone else to the group! I would also like to do some smart-processing of PGP signatures, and the silly markup language that Apple Mail uses. However, it is all far down the track.
A down the road idea ... Is there currently a place in GroupServer's webadmin to define file types to ignore/discard? If there was I'd add VCF to the list, since the value of that attachment is dubious for ongoing group exchanges. http://groups.dowire.org/r/file/3134-2009-04-06T133117Z Name: robin.vcf Tags: "attachment" Type: text/x-vcard Size: 0KB
First progress -- I'm just creating a Xen image at the moment. Been a
while since I've done VM work, bare with me please :)
--Richard
Hi all,
First, this off-list discussion should be move into a GroupServer.org
group, so I have ☺
Second, GroupServer does have some dependencies on specific systems, but
it should not effect what Linux distribution you install it on. I am
mostly familiar with Ubuntu, so I normally phrase my dependencies as
Ubuntu packages:
System Ubuntu Packages
──────────────────────────────────────────────────────────
Subversion subversion
Python python2.4
GNU C++ Compiler g++
Apache Web Server apache2
PostgreSQL Relational Database postgresql
postgresql-server-dev-8.3
Postfix Mail Transfer Agent postfix
postfix-dev
Compression libzzip-dev (from universe)
lib64z1-dev (from main)
Apache and PostgreSQL are needed to run GroupServer. The other
dependencies are needed to install it. I keep the above list up to date
on the GroupServer download page
http://groupserver.org/downloads
GroupServer.org (OnlineGroups, and e-Democracy.org) actually run on
CentOS (a variant of Fedora, like RedHat) while our test platforms run
on Ubuntu. I suspect that installing GroupServer on CentOS, Fedora or
Ubuntu should be fairly straightforward. I would pick Ubuntu purely
because I am familiar with it.
Hope this helps,
Michael
The parameters for Manage Members is very limited. The interface will look the same, and do the same things. The server-side code will be rewritten entirely, and the Manage Bulk Members needs to be removed. By removing Manage Bulk Members you will get your third-option satisfied, because this is how Manage Members has allowed many operations made to many members since I wrote it three years ago. As for option 1 is relatively high on the development adgenda https://svn.iopen.net/projects/groupserver/report/12 https://svn.iopen.net/projects/groupserver/ticket/251 While 2, 4, and 5, I suspect they will have to get in the queue along with all the other development tasks ☺