Enterprise Graph Search

Posted on January 28, 2013 by Christian Ubbesen

Facebook will soon launch their new Graph Search to the general public, and it has received a lot of interest lately.

With graph search, the users will be able to query the social graph that millions of people have constructed over the years when friending each other and putting in more and more personal information about themselves and their friends in the vast Facebook database. It will be possible to query for friends of friends who have similar interests as you, and invite them to a party, or to query for companies where people with similar beliefs as you work, and so on and so forth. The information that is already available, will all the sudden become much more accessible through the power of graph search.

How can we bring this to an enterprise search environment? Well, there are lots of graphs in the enterprise as well to query, both social and other types. For example, how about being able to query for people that have been members of a project in the last three years that involved putting a new product successfully to the market. This would be an interesting list of people to know about, if you’re a marketing director that want to assemble a team in the company, to create a new product and make sure it succeeds in the market.

If we dissect graph search, we will find three important concepts:

The information we want to query against don’t only need to be indexed into one central search engine, but also the relations and attributes of all information objects need to be normalized to create the relational graph and have standard attributes to query against. We could use the Open Graph Protocol as the foundation.
We need a parser that take human language and converts it to a formal query language that a search engine understands. We might want to query in different human languages as well.
The presentation of results should be adapted to the kind of information sought for. In Facebook’s example, if you query for people you will get a list of people with their pictures and some relevant personal information in the result list, and if you query for pictures you will get a collage of pictures (similar to the Google image search).

So the recipe to success is to give the information management part of the project a big focus, making sure to create a unified information model of the content to be indexed. Then create a query parser for natural language based on actual user behavior, and the same user studies would also give us information on how to visualize the different result set types.

I believe we will see more of these kind of solutions in the coming years in the enterprise search market, and look forward exploring the possibilities together with our clients.

Microsoft is betting on cloud, mobile and social for SharePoint 2013 – Impressions from the SharePoint Conference 2012

Posted on January 9, 2013 by Paula Petcu

Over 10,000 attendees from 85 countries, more than 200 sponsors and exhibitors, and over 250 sessions. Besides these impressive numbers, the 2012 SharePoint conference in Las Vegas has also marked the launch of the new version of SharePoint. Findwise was there to learn and is now sharing with you the news about enterprise search in SharePoint 2013.

In the keynote presentation on the first day of the conference, Jared Spataro (Senior Director, SharePoint Product Management at Microsoft) mentions the three big bets made for the SharePoint 2013 product: CLOUD, MOBILE, and SOCIAL. This post tries to provide a brief overview of what these three buzzwords mean for the enterprise search solution in SharePoint 2013. Before reading this, also check out our previous post about search in SharePoint 2013 to get a taste of what’s new in search.

Search in the cloud

While you have probably heard the saying that “the cloud has altered the economics of computing” (Jared Spataro), you might be wondering how to get there. How to go from where you are now to the so-called cloud. The answer for search is that SharePoint 2013 provides a hybrid approach that helps out in this transition. Hybrid search promises to be the bridge between on-premises and the cloud.

The search results from the cloud and those from on-premise can be shown on the same page with the use of the “result blocks”. The result block, new to SharePoint 2013, is a block of results that are individually ranked and are grouped according to a “query rule”. In short, a query rule defines a condition and an action to be fired when the condition is met. With the use of the result blocks, you can display the search results for content coming from the cloud when searching from an on-premises site and the other way around (depending whether you want the search to be one-way or bidirectional), and you can also conditionally enable these result blocks depending on the query (for example, queries matching specific words or regular expressions).

Screenshot from the post Hybrid search of the Microsoft SharePoint Team Blog showing how results from the cloud are integrated in the search results page when the user searches from an on-premises SharePoint 2013 site.

Before making the decision to move to the cloud, it is wise to check the current features availability for both online and on-premise solutions on TechNet.

Mobile devices

With SharePoint 2013, Microsoft has added native mobile apps for Windows, Windows Phone, iPhone, and iPad, and support across different mobile devices (TechNet), which provides access to information and people wherever the users are searching from.

Also important to mention when talking about mobile, is that the improved REST API widens the extensibility options and allows easy development of custom user experiences across different platforms and devices. The search REST API provides access to the keyword query language parameters, and combining this with a bit of JavaScript and HTML allows developers to quickly start building Apps with custom search experiences and making all information available across devices.

Social search

In the same keynote, Jared Spataro said that Microsoft has “integrated social very deeply into the product, creating new experiences that are really designed to help people collaborate more easily and help companies become more agile.” This was also conveyed by the presence of the two founders of the enterprise social network Yammer in the keynote presentation. The new social features integration means that the information about people following content, people following other people, tags, mentions, posts, discussions, are not only searchable but can be used in improving the relevance of the search results and improving the user experience overall. Also, many of the social features are driven by search, such as the recommendations for people or documents to follow.

Whether you are trying to find an answer to a problem to which the solution has already been posted by somebody else, or whether you are trying to find a person with the right expertise through the people search, SharePoint 2013 provides a more robust and richer social search experience than its previous versions. And the possibilities to extend the out-of-the-box capabilities must be very attractive to businesses that are for example looking to combine the social interactivity inside SharePoint with people data stored in other sources (CRM solutions, file shares, time tracking applications, etc).

Stay tuned!

It was indeed an awesome conference, well organized, but most of the times it was hard to decide which presentation to choose from the many good sessions running at the same time. Luckily (or wisely), we had more than one Findwizard on location!

This post is part of our series of reports from the SharePoint 2012 Conference. Keep an eye on the Findability blog for part two of our report from the biggest SharePoint conference of 2012!

Search in SharePoint 2013

Posted on October 18, 2012 by Paula Petcu

There has been a lot of buzz about the upcoming release of Microsoft’s SharePoint 2013, how about the search in SharePoint 2013? The SharePoint Server 2013 Preview has been available for download since July this year, and a few days ago the new SharePoint has reached Release to Manufacturing (RTM) with general availability expected for the first quarter of 2013.

If you currently have an implementation of SharePoint in your company, you are probably wondering what the new SharePoint can add to your business. Microsoft’s catchphrase for the new SharePoint is that “SharePoint 2013 is the new way to work together”. If you look at it from a tech perspective, amongst other features, SharePoint 2013 introduces a cloud app model and marketplace, a redesign of the user experience, an expansion of collaboration tools with social features (such as microblogging and activity feeds), and enhanced search functionality. There are also some features that have been deprecated or removed in the new product, and you can check these on TechNet.

Let’s skip now to the new search experience provided out-of-the-box by SharePoint 2013. The new product revolves around the user more than ever, and that can be seen in search as well. Here are just a few of the new or improved functionalities. A hover panel to the right of a search result allows users to quickly inspect content. For example, it allows users to preview a document and take actions based on document type. Users can find and navigate to past search results from the query suggestions box, and previously clicked results are promoted in the results ranking. The refiners panel now reflects more accurately the entities in your content (deep refiners) and visual refiners are available out-of-the-box. Social recommendations are powered by users’ search patterns, and video and audio have been introduced as new content types. Some of the developers reading this post will also be happy to hear that SharePoint 2013 natively supports PDF files, meaning that you are not required anymore to install a third-party iFilter to be able to index PDF files!

Search Overview in SharePoint 2013, Source: Microsoft SharePoint Team Blog

While the out-of-the-box SharePoint 2013 search experience sounds exciting, you may also be wondering how much customization and extensibility opportunities you have. You can of course search content outside SharePoint and several connectors that allow you to get content from repositories such as file shares, the web, Documentum, Lotus Notes and public Exchange folders are included. Without any code, you can use the query rules to combine user searches with business rules. Also, you can associate result types with custom templates to enrich the user experience. Developers can now extend content processing and enrichment, which previously could have only be achieved using FAST Search for SharePoint. More than that, organizations have the ability to extend the search experience through a RESTful API.

This post does not cover all the functionalities and if you would like to read more about what changes the new SharePoint release brings, you can start by checking the TechNet material and following the SharePoint Team Blog and the Findwise Findability Blog, and then get in touch with us if you are considering implementing SharePoint 2013 in your organization or company.

Findwise will attend the SharePoint Conference 2012 in Las Vegas USA between 12-15 November and this will be a great opportunity to learn more about the upcoming SharePoint. We will report from the conference from a findability and enterprise search perspective. Findwise has years of experience in working with FAST ESP and SharePoint, and is looking forward to discussing how SharePoint 2013 can help you in your future enterprise search implementation.

The Enterprise Search and Findability Report 2012 is ready

Posted on July 6, 2012 by Kristian Norling

No strategy, no budget, no resources. This is the common scenario for enterprise search and findability in many organisations today. Still Enterprise Search is considered a critical success factor in 75% of organisations that responded to the global survey that ran from March to May this year.

The Enterprise Search and Findability Report 2012 is now ready for download.

The Enterprise Search and Findability report 2012 shows that 60% of the respondents expressed that it is very/moderately hard to find the right information. Only 11% stated that it is fairly easy to search for information and as few as 3% consider it very easy to find the desirable information. This shows that there still is a large untapped potential for any organisation to get great value from investing in enterprise search. For a relatively small investment, preferably in personnel it is possible to make search a lot better. The survey also reveals that organisations who are very satisfied with their search, have a (larger) budget, more resources and systematically work with analysing search.

Figure. What is your primary goal for utilising search technology in your organisation?

The primary goal for using search is to accelerate retrieval of known information sources, 91%, and to improve the re-use of content (information/knowledge), 72%. This indicates that often search within organisations is used as a discovery tool for what already is known. If looking over the next three years, as many as 77% think that the amount of information in the organisation will increase. This means that every year it will be even more important be able to find the right information and that means Enterprise search is still very much needed, as stated in the following great presentations (on video): Why Business Success Depends on Enterprise Search (by Martin White of Intranet Focus) and The Enterprise Search Market – What should be on your radar? (by Alan Pelz-Sharpe of 451 Research)

Download the full report.

Findability day in Stockholm – search trends and customer insights

Posted on June 26, 2012 by Caroline Abrahamsson

Last Thursday about 50 of Findwise customers, friends and people from the industry gathered in Stockholm for a Findability day (#findday12). The purpose was simply to share experiences from choosing, implementing and developing search and findability solutions for all types of business and use cases.

Martin White, who has been in the intranet business since 1996, held the keynote speech about “Why business success depends on search”.
Among other things he spoke about why the work starts once search is implemented, how a search team should be staffed and what the top priority areas are for larger companies.
Martin has also published an article about Enterprise Search Team Management that gives valuable insight in how to staff a search initiative. The latest research note from Martin White on Enterprise search trends and developments.

Henrik Sunnefeldt, SKF, and Joakim Hallin, SEB, were next on stage and shared their experiences from working with larger search implementations.
Henrik, who is program manager for search at SKF, showed several examples of how search can be applied within an enterprise (intranet, internet, apps, Search-as-a-Service etc) to deliver value to both employees and customers.
As for SEB, Joakim described how SEB has worked actively with search for the past two years. The most popular and successful implementation is a Global People Search. The presentation showed how SEB have changed their way of working; from country specific phone books to a single interface that also contains skills, biographies, tags and more.

During the day we also had the opportunity to listen to three expert presentations about Big data (by Daniel Ling and Magnus Ebbeson), Hydra – a content processing framework – video and presentation (by Joel Westberg) and Better Business, Protection & Revenue (by David Kemp from Autonomy).
As for Big data, there is also a good introduction here on the Findability blog.

Niklas Olsson and Patric Jansson from KTH came on stage at 15:30 and described how they have been running their swift-footed search project during the last year. There are some great learnings from working early with requirements and putting effort into the data quality.

Least, but not last, the day ended with Kristian Norling from Findwise who gave a presentation on the results from the Enterprise Search and Findability Survey. 170 respondents from all over the world filled out the survey during the spring 2012 that showed quite some interesting patterns.
Did you for example know that in many organisations search is owned either by IT (58%) or Communication (29%), that 45% have no specified budget for search and 48% of the participants have less than 1 dedicated person working with search? Furtermore, 44,4% have a search strategy in place or are planning to have one in 2012/13.
The survey results are also discussed in one of the latest UX-postcasts from James Royal-Lawson and Per Axbom.

Thank you to all presenters and participants who contributed to making Findability day 2012 inspiring!

We are currently looking into arranging Findability days in Copenhagen in September, Oslo in October and Stockholm early next spring. If you have ideas (speakers you would like to hear, case studies that you would like insight in etc), please let us know.

Reflections on Search at Intranets 2012 conference

Posted on May 24, 2012 by Mattias Brunnert

Despite large corporations spending hundreds of millions of euros creating information they spend almost nothing on search, Martin White said at the recent Intranets 2012 conference. But before dealing with this depressing fact, I would like to start on a more positive note.

Being a search professional it was an absolute joy to jump over to the other side of the fence and join the well over hundred intranet professionals at intranets 2012 in gorgeous Sydney. I whole heartedly recommend to search #intranets2012 on twitter to get a feel for the fun, inspiration and knowledge sharing that went on.

With sessions on collaboration, from recognized experts such as Michael Sampson, or by seasoned practitioners such as William Amurgis from American Electric Power, it was clear that social intranets are not only a buzz word but are already providing businesses with great value. Meanwhile James Robertson demanded that we raise the bar for design and usability from providing function to delivering pretty and simple intranets that surprise and delight. Mandy Geddes from Institute of Executive coaching gave me a brilliant idea of how to use private online communities to engage customers.

But in spite of returning from Sydney with a feeling of new energy, eagerness and almost urgency to get back to helping my customers and colleagues, I also realized that search was obviously not on everyone’s mind. Except for Martin White‘s excellent keynote only one session I attended to, Ausgrid Power‘s presentation of their intranet “the grid”, had search as a key area. Hopefully these few glimpses of light sparked something and I honestly think they do, bearing in mind the discussions I had in the breaks and in the fantastic social event Thursday evening.

After writing this to share my thinking, I have two things to say:

Findability ambassadors; our work has only begun and a I hope to see you all at Intranets 2013, because I’m sure going!

Data and Search Going Big?

Posted on April 25, 2012 by Martin Johansson

A few enterprise search specialists from Findwise recently attended the Scandinavian Developer Conference 2012. One of the tracks was Big Data, which is very much related to search. It had some interesting talks about how to handle large amounts of data in an efficient way. Special thanks to Theo Hultberg, Jim Webber and Tim Berglund!

The theme was that you should choose a storage system which is well suited for the task. This may seem like an obvious point, but for a long time this was simply ignored; I’m talking about the era of relational databases. Don’t get me wrong, sometimes a relational database is the very best for the job, but in many cases it isn’t.

Data is jagged by nature, i.e. not all objects have the same properties. This is why we shouldn’t force them to fit into a square table, instead everything should be denormalized! The application accessing the data will be aware of the information structure and will handle it accordingly. This will also avoid expensive assembly operations (such as joins) to get the data in the format we want when retrieving it. Why should you split up your data if you are going to assemble it over and over again? Also remember that disk space is cheap, pre-compute as much as possible. The design of a Big Data system should be governed by how the data will be retrieved.

Another step away from the relational databases is the relaxation of some of the ACID properties: Atomicity, Consistency, Isolation and Durability. Again, this is along the lines of choosing the components best suited for the system. Decide which properties are a must have and which are not so important.

Relaxing the ACID properties, such as consistency, can give great performance gains. The NoSQL database Cassandra is eventually consistent and its write performance scales linearly up to 288 nodes (and probably even higher) which gives a write performance of over 1 million writes per second!

However, relaxation of these properties is not a new concept in the world of search engines. When indexing a document, it will usually take a number of seconds before it is searchable. This is called eventual consistency, i.e. the state of the search engine will be brought from one valid state to another, within a sufficiently long period of time. Do we really need documents that were just submitted to the search engine to be
searchable instantly? Most likely, no. Isolation is another property that is not crucial to a search engine. Since a document in an index doesn’t have any explicit relations to any other documents in the same index, there isn’t a great need for isolation. If two writes for the same document are submitted at the same time, there is probably something wrong in another part of the system.

So what does all this mean for search? There is an interesting challenge in storing jagged data in large amounts and then making good use out of it. To search in vast amounts jagged data, you need a lot of querytime field mappings (to make relevant data searchable) … or do you? There is also the issue of retaining a good relevancy model, which is absolutely vital to a search engine. How do you measure the relevance of arbitrary metadata and then weigh it all together? Maybe we need to think in new ways about relevance all together?

Whomever can solve these problems in a good way with a minimum amount of manual labor, is a name we’ll be hearing from a lot in the future.

Update on The Enterprise Search and Findability Survey

Posted on April 12, 2012 by Kristian Norling

A quick update on the status of the Enterprise Search survey.

We now have well over a hundred respondents. The more respondents the better the data will be, so please help spreading the word. We’d love to have several hundred more. The survey will now be open until the end of April.

But most important of all, if you haven’t already, have a cup of coffee and fill in the survey.

A Few Results from the Survey about Enterprise Search

More than 60% say that the amount of searchable content in their organizations today are less or far less than needed. And in three years time 85% say that the amount of searchable content in the organisation will increase och increase significantly.

75% say that it is critical to find the right information to support their organizations business goals and success. But the interesting to note is that over 70% of the respondents say that users don’t know where to find the right information or what to look for – and about 50% of the respondents say that it is not possible to search more than one source of information from a single search query.

In this context it is interesting that the primary goal for using search in organisations (where the answer is imperative or signifact) is to:

Improve re-use of information and/or knowledge) – 59%
Accelerate brokering of people and/or expertise – 55%
Increase collaboration – 60%
Raise awareness of “What We Know” – 57%
and finally to eliminate siloed repositories – 59%

In many organisations search is owned either by IT (60%) or Communication (27%), search has no specified budget (38%) and has less than 1 dedicated person working with search (48%). More than 50% have a search strategy in place or are planning to have one in 2012/13.

These numbers I think are interesting, but definitely need to be segmented and analyzed further. That will of course be done in the report which is due to be ready in June.

Enterprise Search Stuffed up with GIS

Posted on February 3, 2012 by Pawel Wroblewski

When I browsed through marketing brochures of GIS (Geographic Information System) vendors I noticed that the message is quite similar to search analytics. It refers in general to integration of various separate sources into analysis based on geo-visualizations. I have recently seen quite nice and powerful combination of enterprise search and GIS technologies and so I would like to describe it a little bit. Let us start from the basic things.

Search result visualization

It is quite obvious to use a map instead of simple list of results to visualize what was returned for an entered query. This technique is frequently used for plenty of online search applications especially in directory services like yellow pages or real estate web sites. The list of things that are required to do this is pretty short:

- geoloalization of items – it means to assign accurate geo coordinates to location names, addresses, zip codes or whatever expected to be shown in the map; geo localization services are given more less for free by Google or Bing maps.

- backgroud map – this is necessity and also given by Google or Bing; there are also plenty of vendors for more specialized mapping applications

- returned results with geo-coordinates as metadata – to put them in the map

Normally this kind of basic GIS visualisation delivers basic map operations like zooming, panning, different views and additionally some more data like traffic, parks, shops etc. Results are usually pins [Bing] or drops [Google].

Querying / filtering with the map

The step further of integration between search and GIS would be utilizing the map as a tool for definition of search query. One way is to create area of interest that could be drawn in the map as circle, rectangle or polygon. In simple way it could be just the current window view on the map as the area of query. In such an approach full text query is refined to include only results belonging to area defined.

Apart from map all other query refinement tools should be available as well, like date-time sliders or any kind of navigation and fielded queries.

Simple geo-spatial analysis

Sometimes it is important to sort query results by distance from a reference point in order to see all the nearest Chinese restaurant in the neighborhood. I would also categorize as simple geo-spatial analysis grouping of search result into a GIS layers like e.g. density heatmap, hot spots using geographical and other information stored in results metadata etc.

Advanced geo-spatial analysis

More advance query definition and refinement would involve geo-spatial computations. Basing on real needs it could be possible for example to refine search results by an area of sight line from a picked reference point or select filtering areas like those inside specific borders of cities, districts, countries etc.

So the idea is to use relevant output from advanced GIS analysis as an input for query refinement. In this way all the power of GIS can be used to get to the unstructured data through a search process.

What kind of applications do you think could get advantage of search stuffed with really advanced GIS? Looking forward to your comments on this post.

Gartner and the Magic Quadrants – Crowning the Leaders of Enterprise Search

Posted on January 25, 2011 by Caroline Abrahamsson

For years Gartner, the research and advisory company, has been publishing their magic quadrants – and their verdict of everything from ECM-systems to Data Warehouse and E-commerce plays a big role in many company’s decision to choose the right tools.
Simply put, the vendors are presented in a matrix measuring the different players by ability to execute (product, overall viability, customer experience etc.) and the completeness of their vision (offering strategy, innovation etc.). The vendors are then positioned as niche players (a rather crowded spot), visionaries, challengers and leaders.

At the end of last year Gartner decided to retire their old “Information Access Quadrant” (Enterprise Search Quadrant) and introduce “Enterprise Search MarketScope” due to a more mature market. A number of vendors (such as Vivisimo and Recommind) were removed, in order to exclude those whose businesses were not entirely search driven.

The evaluation criteria’s for MarketScope cover: offering (product) strategy, Innovation, Overall viability (business unit, financial, strategy, and organization), Customer experience, Market understanding and business model.

To summarize: the criteria’s are to a large extent the same, but the two areas “overall viability” and “customer experience” are weighted higher than the rest. This is most likely a result of the last years discussion around user friendly interfaces, easier administration and the fact that some customers have suffered quite bad when vendors do not survive (one example in Northen Europe is the Danish vendor that went bankrupted for some time)

The yearly fight between the three leaders; Microsoft, Endeca and Autonomy has been somewhat disrupted and Microsoft, Endeca and Google are now seen as the leaders.
Microsoft has got a very broad product line, which stretches from low-price and less functionality to Enterprise Search built on the former FAST technology. Endeca follow the same trend, as Gartner puts it their “products (are) intended to serve organizations seeking to develop general search installations..(..) broadly applicable for a variety of different search challenges”.

In the old quadrant, Google remained a “challenger” for quite some time – but never made it to the “leaders” corner. Ease of administration and “user friendly” are two words that keeps being repeated. That, in combination with a profit of $ 7290000000 during the last quarter of 2010 makes Google a player that easily can continue to develop their Enterprise business.

Gartner’s MarketScope for Enterprise Search

Autonomy should still not be disregarded, the main reason for it falling a bit behind the three others seem to be conquerable problems with support and pricing transparency. It will be interesting to see how Autonomy chooses to handle these issues during 2011.

To put it short: the new MarketScope is good reading with quite few surprises. If you wish to get a better understanding of the development going on at the different vendors, start with Gartner and continue to search among our blog posts.

The Findability blog

the enterprise search and findability blog by Findwise

Category Archives: Search Watch