Presentation made by @kristiannorling at IntraTeam 2012, 1st of March in Copenhagen.
View more presentations from Findwise
Search Analytics in Practice
Reply
Category Archives: Internet search
European Conference on Information Retrieval (ECIR) 2011 in retrospect
The European Conference on Information Retrieval (ECIR) 2011 took place in Dublin last week, 18-21 April. In this blogpost I would try to highlight some of the papers and talks from the conference which caught my attention and back it up with what other attendees said about it.
First, I was intrigued by the session on evaluation for IR and especially the topic of Croudsourcing. In my opition, the paper A Methodology for Evaluating Aggregated Search Results, which also got the prize for best student paper, was among the most pedagogically presented ones. It deals with the task of incorporating search results from a number of different sources, called verticals, into Web search results. By using a small number of human judgements for a given query the authors present the way to evaluate any possible permutation of verticals in the result presentation. I think that this methodology should be adopted in the world of Enterprise search, since it is exactly there where we crawl, index and present information from a number of different sources – Web, databases, fileshares, etc. The prerequisites are really minimal and low cost but the return value, the user experience, seems quite high.
Amazon Mechanical Turk, or the Artificial Artificial Intelligence, which is the marketplace for Croudsourcing, provides a way for a ridiculously small sum of money to perform evaluation, relevance assessment or any task for which you would need humans to give you some judgements. Leaving aside ethical issues, two papers in the conference presented ways of how you can utilize this service for some IR tasks.
Evgeniy Gabrilovich from Yahoo! Research, who won the Karen Sparck Jones award for 2010, gave a very interesting keynote talk on Computational Advertising. Up to now, it has never struck me how hard advertising in Information Retrieval systems is actually. I liked one of his points on the future of Ads – by using product feeds, one can automatically create product description via Text Summarization and Natural Language Generation and index this, thus avoiding bid words.
Another interesting and very pedagogically presented paper was about the gensim package by Radim Řehůřek. I definitely think we can use it in some of our projects. In general, text categorization and IR for social network were the dominant tracks. In one of the social networks tracks, Oscar Täckström presented a neat way of discovering fine-grained sentiment where some coarse-grained supervision is available. It really hooked me on trying it for any of our customers where sentiment analysis is required.
Thorsten Joachims, the last of the keynote speakers, gave a very inspiring talk on The Value of User Feedback. He put forward the idea of designing retrieval systems for feedback. In stead of just looking at the clicklogs post factum one can think of a system which uses the clicks feedback to learn, thus creating a better ranker for a given query and a given user need. In a single session, we can use click feedback to disambiguate the query and deliver results on the run which are of immediate benefit to the users.
Unfortunately, I guess I could have missed other interesting presentations but with two parallel sessions and several workshops there was a limit to what I could devour. What surprised me though, was that there were very few papers by the industry. We do try to solve exactly the same problems and tackle the same issues as academia. We, at Findwise, have constantly flagged the huge benefit of good, relevant Metadata for the task of achieving better search performace, which was also touched upon in the paper “Topic Classification in Social Media using Metadata from Hyperlinked Objects”.
It was really great to visit Dublin and attent ECIR 2011. It was an inspiring conference and I do believe that at next ECIR we, from Findwise, can be on the podium, sharing our knowledge and hands-on experience on Enterprise search and IR.
Sláinte!
Google Instant – Can a Search Engine Predict What We Want?
On September 8th Google released a new feature for their search engine: Google instant.
If you haven’t seen it yet, there is an introduction on Youtube that is worth spending 1:41 minutes on.
Simply put, Google instant is a new way of displaying results and helping users find information faster. As you type, results will be presented in the background. In most cases it is enough to write two or three characters and the results you expect are already right in front of you.
The Swedish site Prisjakt has been using this for years, helping the users to get a better precision in their searches.
At Google you have previously been guided by “query suggestion” i.e. you got suggestions of what others have searched for before – a function also used by other search engines such as Bing (called Type Ahead). Google instant is taking it one step further.
When looking at what the blog community has to say about the new feature it seems to split the users in two groups; you either hate it or love it.
So, what are the consequences? From an end-user perspective we will most likely stop typing if something interesting appears that draws our attention. The result?
The search results shown at the very top will generate more traffic , it will be more personalized over time and we will most probably be better at phrasing our queries better.
From an advertising perspective, this will most likely affect the way people work with search engine optimization. Some experts, like Steve Rubel, claims Google instant will make SEO irrelevant, wheas others, like Matt Cutts think it will change people behavior in a positive way over time and explains why.
What Google is doing is something that they constantly do: change the way we consume information. So what is the next step?
CNN summarizes what the Eric Schmidt, the CEO of Google says:
“The next step of search is doing this automatically. When I walk down the street, I want my smartphone to be doing searches constantly: ‘Did you know … ?’ ‘Did you know … ?’ ‘Did you know … ?’ ‘Did you know … ?’ ”
Schmidt said at the IFA consumer electronics event in Berlin, Germany, this week.
“This notion of autonomous search — to tell me things I didn’t know but am probably interested in — is the next great stage, in my view, of search.”
Do you agree? Can we predict what the users want from search? Is this the sort of functionality that we want to use on the web and behind the firewall?
Findability in Customer Service Search
We have previously introduced Findability by Findwise, involving solutions that make optimal use of search technology to support and strengthen the business of our customers. In a series of blog posts we will present how findability solutions can be deployed within different parts of your organisation. Initially I will focus on how efficient implementation of search technology, by a good customer service search, can improve your customer service offering.
Ultimately, the goal of most customer service interactions is to increase customer satisfaction and thereby improve customer retention in a cost efficient way. In times when the amount of available information increases by the minute, one key success factor is to provide both customer service agents and customers with quick and easy access to relevant information. A findability solution based on state-of-the-art search technology and optimised along the findability dimensions will fuel your customer service search offering in two primary ways:
- Improved support to customer service agents
- Improved online customer service
Improved support to customer service agents
While more traditional customer service interaction solutions tend to be based on a knowledge database, that needs to be built and maintained, a Findability solution is more dynamic in its nature and is based on a dynamic search index created by the already existing data residing in corporate systems. In other words, the solution makes optimal use of existing information and systems to support customer service agents in accessing relevant information. The positive effects are illustrated by the case study below.
Case study: Telecom call centre
Findwise implemented a findability solution at a call centre for a large Swedish mobile operator. The solution introduced the powerful ability to search in the most important information source, which previously only had been accessible via tree-structure navigation.
The graph below presents the result of a test performed by the call centre agents to evaluate the new search function. The test encompassed a number of tasks in which the agents compared using the search functionality to the traditional navigation, in terms of both level of difficulty and time consumption in finding desired information. The graph shows that the agents found the search function very helpful, making the information both easier and less time consuming to find.
The most evident effects of improved support and information access via search technology are:
- Reduced handling time
- Higher first time resolution
- Reduced Tier-2 escalations
- Increased customer service agent satisfaction
- Increased agent productivity
- Less training needed to introduce new agents
In a white paper, Google has also pinpointed, and quantified, the above benefits of implementing a Findability solution in call centre operations, in this case fuelled by the Google Search Appliance (GSA) search platform. For example, Google states that handling time can be reduced by up to 20% on average and that is it possible to save up to 25% on training costs for each new call centre agent. The full article is available here.
Improved online customer service
Naturally a Findability solution can also improve your online customer service offering. Below I have outlined three solution elements that will help drive customer self-service and thereby deflect issues from being forwarded to the customer service organisation.
Improved search functionality
As in the case of agent support, a powerful search functionality that provides relevant information from all required sources in a user-friendly way will increase the ability of customer self-resolution.
Personalised user interface
Using the power of an enterprise search platform you can customise the self-service experience, in a dynamical way, to the individual and the incident to simplify and speed up the process of finding answers.
Dynamic FAQ
Self-service can also be fuelled by providing a relevant and updated FAQ section. The information can be made dynamic and include answers to the most recent questions by using both query log information, i.e. what users are searching for, and call centre comments as input to the FAQs.
For many enterprises, self-service is seen as the solution that can provide customers with the support they need while significantly reducing customer service costs. However, self-service must do more than just cut costs. When customers perceive self-service as simply a means to shift interaction costs onto their shoulders, it can reduce customer satisfaction. Customers need a self-service experience that provides them with higher levels of interaction convenience and information availability, faster issue resolution and more personalised interactions. A Findability solution including the above elements provides that.
The most evident effects of an improved online customer service offering gained from the use of search technology and search analytics are:
- Less number of incoming calls/e-mails
- Increased customer satisfaction
- Increased browser- to-buyer conversion rate
- Increased knowledge of user interests and behaviour (to fuel additional sales)
Visit our website to learn more about findability solutions that make our customers truly benefit from state-of-the-art search technology.
Structured and actionable search results – there is more to presentation than blue links
Search patterns are standardized patterns describing search functionality as well as human information seeking behavior. Earlier this year Peter Morville and Jeffery Callender released a book about search patterns. Morville also gave a presentation based on the book at the IA Summit 2010 (slides, mp3), which my colleague Maria and I attended. Among the patterns Peter Morville mentions, my favorite ones are structured and actionable search results.
Structured search results
Let us start with structured search results. You might have seen that for certain queries you submit on Google, you get a richer results presentation than for other results. For example, typing the query ‘weather stockholm’ gives a basic weather forecast for the upcoming four days, directly visible in the results list. Other examples include local movie showtimes and stock information. It is even possible to use google as a calculator or a currency converter by typing in certain kinds of searches. For the curious, here is a list of all google.com search features. Structured results is about offering a more informative presentation of search results than just a title, summary, and possibly some basic metadata. It is also about not presenting all information in the same way, because the information in itself differs. Richer results presentations speeds up the process of finding relevant information since the system has already done some pre-processing for user.
Structured metadata is a prerequisite for structured search results presentation. Web pages and documents normally come with standard metadata such as date and author, but in some cases they will have to be augmented with additional information in order to create a more useful presentation. Presenting results in a custom way requires some extra development effort, especially if the structure is not initially available. However, I believe it creates much value to the user. Also, this need not be done for all types of contents. My advice would be to identify the cases where a more elaborate results presentation would be most usable. Which information is frequently requested by many people and perhaps also difficult to find because it is embedded in pages with lots of text or other contents? Search logs and user feedback in combination with thorough knowledge about the contents provides a key basis for the selection.
Actionable search results
Related to structured results are actionable search results. Entries in the search results list can be more than just displays of information; they can also be means of performing tasks. Common examples found on the web include printing, saving or sharing the search result directly from the results list. Other examples include adding to shopping cart, commenting and rating. Within the enterprise or organization additional relevant actions could perhaps be checking in or out a document, add an event to the personal calendar, starting a chat with a co-worker, and so on. As with structured results, it is about identifying the cases where it would add most value. What are the most common tasks and possibly also what tasks are complicated to perform in the source system? Structured and actionable results share the advantage that users do not have to open the actual results web page or download the document to find or do what they need. Speeding up information seeking and other tasks in this way is not only valuable in web search, it can also be very useful within the enterprise or organization. Search results lists in enterprise search solutions still look quite homogeneous and there are lots of opportunities for improvement.
To conclude, there can and should be more to search results presentation than just a snippet. I believe we will benefit from putting focus on the results presentation, and not only on tools surrounding it (filtering for example). After all, the list of results is where the user’s attention is first drawn. What do you think? How can your organization benefit from working with structured and actionable search results? If you are curious about this approach, we would be happy to help you look into what can be done in your organization.
Search in SharePoint 2010
This week there has been a lot of buzz about Microsoft’s launch of SharePoint 2010 and Office 2010. Since SharePoint 2007 has been the quickest growing server product in the history of Microsoft, the expectations on SharePoint 2010 are tremendous. And also great expectations for search in Sharepoint 2010
Apart from a great deal of possibilities when it comes to content creation, collaboration and networking, easy business intelligence etc. the launch also holds another promise: that of even better capabilities for search in Sharepoint 2010 (with the integration of FAST).
Since Microsoft acquired FAST in 2008, there have been a lot of speculations about what the future SharePoint versions may include in terms of search. And since Microsoft announced that they will drop their Linux and UNIX versions in order to focus on higher innovation speed, Microsoft customer are expecting something more than the regular. In an early phase it was also clear that Microsoft is eager to take market shares from the growing market in internet business.
So, simply put, the solutions that Microsoft now provide in terms of search is solutions for Business productivity (where the truly sophisticated search capabilities are available if you have Enterprise CAL-licenses, i.e. you pay for the number of users you have) and Internet Sites (where the pricing is based on the number of servers). These can then be used in a number of scenarios, all dependent on the business and end-user needs.
Microsoft has chosen to describe it like this:
- “Foundation” is, briefly put, basic SharePoint search (Site Search).
- “Standard” adds collaboration features to the “Foundation” edition and allows it to tie into repositories outside of SharePoint.
- “Enterprise ” adds a number of capabilities, previously only available through FAST licenses, such as contextual search (recognition of departments, names, geographies etc), ability to tag meta data to unstructured content, more scalability etc.
I’m not going to go into detail, rather just conclude that the more Microsoft technology the company or organization already use, the more benefits it will gain from investing in SharePoint search capabilities.
And just to be clear: non-SharePoint versions (stand-alone) of FAST are still available, even though they are not promoted as intense as the SharePoint ones.
Apart from Microsoft’s overview above, Microsoft Technet provides a more deepdrawing description of the features and functionality from both an end-user and administrator point of view.
We look forward describing the features and functions in more detail in our upcoming customer cases. If you have any questions to our SharePoint or FAST search specialist, don’t hesitate to post them here on the blog. We’ll make sure you get all the answers.
Search and Accessibility
Västra Götalands regionen has introduced a new search solution that Findwise created together with Netrelations. Where both search and accessibility is important. We have also blogged about it earlier (see How to create better search – VGR leads the way). One important part of the creation of this solution was to create an interface that is accessible to everyone.
Today the web offers access to information and interaction for people around the world. But many sites today have barriers that make it difficult, and sometimes even impossible for people with different disabilities to navigate and interact with the site. It is important to design for accessibility – so that no one is excluded because of their disabilities.
Web accessibility means that people with disabilities can perceive, understand, navigate, interact and contribute to the Web. But web accessibility is not only for people that use screen readers, as is often portrayed. It is also for people with just poor eyesight who need to increase the text size or for people with cognitive disabilities (or sometimes even for those without disabilities). Web accessibility can benefit people without disabilities, such as when using a slow Internet connection, using a mobile phone to access the web or when someone is having a broken arm. Even such a thing as using a web browser without javascript because of company policy can be a disability on the web and should be considered when designing websites.
So how do you build accessible websites?
One of the easiest things is to make sure that the xhtml validates. This means that the code is correct, adheres to the latest standard from W3C (World Wide Web Consortium) and that the code is semantically correct i.e. that the different parts of the website use the correct html ”tags” and in the correct context. For example that the most important heading of a page is marked up with ”h1” and that the second most important is ”h2” (among other things important when making websites accessible for people using screen readers).
It is also important that a site can easily be navigated only by keyboard, so that people who cannot use a mouse still can access the site. Here it is important to test in which order the different elements of the web page is selected when using the keyboard to navigate through the page. One thing that is often overlooked is that a site often is inaccessible for people with cognitive disabilities because the site contains content that uses complex words, sentences or structure. By making content less complex and more structured it will be readable for everyone.
Examples from VGR
In the search application at VGR elements in the interface that use javascript will only be shown if the user has a browser with java script enabled. This will remove any situations where elements do not do anything because java script is turned off. The interface will still be usable, but you will not get all functionality. The VGR search solution also works well with only the keyboard, and there is a handy link that takes the user directly to the results. This way the user can skip unwanted information and navigation.
How is accessibility related to findability?
http://www.flickr.com/photos/morville/4274260576/in/set-72157623208480316/
Search and Accessibility
Accessibility is important for findability because it is about making search solutions accessible and usable for everyone. The need to find information is not less important if you are blind, if you have a broken arm or if you have dyslexia. If you cannot use a search interface you cannot find the information you need.
“what you find changes who you become” -Peter Morville
In his book Search Patterns Peter Morville visualizes this in the ”user experience honeycomb”. As can been seen in the picture accessibility is as much a part of the user experience as usability or findability is and a search solution will be less usable without any of them.
How to Create Better Search – VGR leads the way
I realise we are a bit late. Fredrik Wackå, a senior IT-strategist, has already written an excellent article on his blog (in Swedish). He has, among other things, been interviewing Kristian Norling (at Twitter), who has been working with portal strategies and better search for many years at Region Västra Götaland. Although, for all our non-Swedish speaking guests here is a short summary:
Findwise has during the last few months been working on a new search solution for Region Västra Götaland. The two main goals have been to deliver a search experience that seems both fast and accurate. The result? Today making a search at VGR takes about 0,1-0,2 seconds, faster than a Google search on the web.
Furthermore, there was a need for context. Large amount of information requires ways to filter and sort – otherwise the users will drown in the result list. By giving the end-users the ability to sort the search result the users can look for general information within an area as well as quickly narrow down to a specific piece (for example by two clicks be able to see only the PDF-files created in 2009). The filters (and thereby metadata standard) includes:
- Information type
- Where the document resides
- Where it belongs in the organization
- What source it has
- When it was last changed
- Who has written it
- What format it resides in
- Keywords that has been created
The search solution also includes a metadata service. As so many others VGR has been struggling with getting the metadata in place.
Apart from the metadata supported by the system (where Dublin Core is being used) the metadata service is doing two things:
- Analyses the content in the text, compares it to taxonomy and gives the writer suggestions of keywords that he/she can use
- Gives the writer the ability to add additional keywords
Apart from this the end-users will be able to add etiquettes (tags). These will be compared with two lists. If the tags appears in the “white list” it will be published right away, if they are in the “blacklist” they will be deleted. Anything inbetween are controlled before they are published.
To conclude: a lot of effort has been put into creating a good search experience and VGR continues to deliver functionality and solutions that are light-years ahead of many others. The combination of supporting systems and using the “collected intelligence” of the writers and end-users will make it even better over time. Search is about both supporting systems, content and people.
Read more in Fredrik Wackås blog
Do You Know Something I Don’t? The Art of Benchmarking Enterprise Search
During the autumn we have been trying to keep our customers and others up to date with the search world by hosting breakfast seminars. By benchmarking enterprise search and sharing experiences and discussing with others the participants have taken giant leaps in understanding what search can deliver in true value. The same goes for sharing experiences between companies, where you often find yourself struggling with the same problems, regardless of business or company size.
We have been discussing how enterprise search can help intranets, extranets, web sites and support centers to capitalize on their knowledge. Some of the things that have been discussed in regards to benchmarking enterprise search.
Business Cases
- How can search help companies save 100 million SEK/year?
- How do you count return on investment (ROI) for search?
Search Functionality
How and why should you work with:
- Key Matches to promote certain content (similar to Google’s sponsored links on the web)
- Synonyms (to make sure that the end-users language corresponds to the corporate without having to change the information)
- Query completion and suggestion to give the user an overview of what other people have been searching for when they start to type (similar to Apples web site search).
End User Experience
- How can different interfaces serve different information needs and user-groups?
- How does your user interface serve your end-users?
Information Quality
- Do taxonomies and folksonomies help us find information faster?
- Can search be used to improve the quality of your content?
During the spring we will continue to hold seminars, keeping you up-to date. If you’re not on our mailing list, please look at our Findability Events and register for our events.
During Wednesday and Thursday this week we will be attending a Ability conference to discuss search. Hope to see you there!
The Future of Information Discovery
I recently attended the third annual workshop on Human Computer Interaction and Information retrieval ( HCIR 2009) in Washington DC together with my colleague Lina. This is the first in a series of blog posts about what happened at the workshop. First up is the keynote about the Future of Information Discovery, by Ben Shneiderman.
Ben Shneiderman, professor at the University of Maryland and founding director of the Human Computer Interaction Laboratory held the workshop keynote. He started off by talking about what he called the elephant in the room, Google. Because whenever you talk about search these days you have to talk about Google. Google has become the baseline for search and the system that users relate other search experiences to. Almost all of our customers’ users has in one way or another asked “why can’t our intranet be more like Google?” (Read more about expectations to Googlify the company in a previous blog post by Mickel. You can also download the slides to Ben Shneidermans keynote presentation.)
As Ben Shneiderman said, Google does actually do the job, finding facts work. However searching for information can be dangerous. Google does well on handling simple fact-finding tasks but we need better tools to handle other types of searches such as:
- Extended fact finding tasks where the queries are often vague
- Tasks involving exploration of availability where the requested results can be vague
- Open ended browsing and problem analysis where there can be hidden assumptions
- Mismatch between the users information needs and the available metadata which will require exhaustive searching.
One of the points that I appreciated the most in this keynote was that systems that support searching for information not only need to support simple known-item searches, which Google does well. They also need to support other things:
- Helping users enrich query formulation
- Expanding result management
- Enable long-term effort
- Enhance collaboration
I am especially pleased by this statement since these are some of the important issues that we are working with in our customer projects. You will also learn more about query formulation in one of our upcoming blog posts from HCIR.
Supporting these cases are important for supporting users in their information seeking tasks and, according to Shneiderman, this should also be done while enabling users to deal with specific cases of search, concerning:
- Completeness – Do I have all the information on a specific topic? This is especially important in for example legal or medical cases.
- Absence of information – proving non-existence of information is very difficult but needed when applying for a patent or registering a trademark.
- Outliers – making unexpected connections between information and finding and learning new things that you would not have expected to find.
- Bridging – Connecting different disciplines with each other.
This is very important because when users search the goal is not the information itself. No users go to a search interface just for the fun of searching for information. They need the information for a purpose. Search therefore needs to support things such as decision-making, collaboration, innovation and societal improvement. Search will only be of true value to users when it not only searches the simple fact-finding tasks but when it helps users solve the real problems in the real world. And good tools can force people to reframe their thinking and see things in a different light. That is the kind of tools that we should be designing.