Search Web Parts – Collab365 Global Conference

 

Have you heard about the virtual Collab365 Global Conference 2017 that’s streaming online November 1st – 2nd?

Join me and 120 other speakers from around the world who will be bringing you the very latest content around SharePoint, Office 365, Flow, PowerApps, Azure, OneDrive for Business and of course the increasingly popular Microsoft Teams. The event is produced by the Collab365 Community and is entirely free to attend.

Places are limited to 5000 so be quick and register now.

During the conference I'd love you to watch my session which is called : 'Search Web Parts'

Content Search Web Part (CSWP) is one of the great web parts in O365 and on-premises. In this session, Mike will demo how to configure a​nd use the CSWP, and build a dynamic O365 branded portal with CSWP only. In this session, we will review: 1.Creating Queries using Keyword Query Language (KQL) 2.Building dynamic queries 3.Creating and customizing HTML Display Templates​​.​

If you join me, you will learn:

  1. Creating Queries using Keyword Query Language (KQL)
  2. Building dynamic queries
  3. Creating and customizing HTML Display Templates
  4. Building Search Driven Portals

Topic(s):

  1. Office365
  2. SharePoint

Audience :

  1. IT Pro

Time (in UTC) :

  1. Thursday, November 2 2017 5:00 PM

How to attend :

  1. Register here.
  2. At the time listed above go here to watch my session. (you can also add me to your own personal planner from the agenda.
  3. Be ready to take notes!

SharePoint 2013 Search Crawl Timeout Issue

​Implementing SharePoint 2013 in a secure zone as an extranet application might be challenging, if you are deploying your farm in a zone with many restrictions.

Recently, I deployed a large SharePoint 2013 farm in a DMZ zone for a regulated portal. Regulated data in my case meant the following restrictive rules in the network and on the servers in the farm:

  • Strict GPO Policies
  • WFE, Application, Search, and SQL servers are hosted in different subnet zones
  • Everything is blocked on the firewall unless specific ports are requested to be open
  • Outbound internet access is disabled on all servers.

Configuring SharePoint in this environment was not a straight forward exercise. After disabling some GPO policies to allow the creation of the IIS web applications, we had to map out the communication between all the servers so the firewall ports are open, allowing each server in the farm to talk to each other.

To get a better understanding of the ports required in your farm, you can follow this TechNet article. It explains the details of each port and its use.

Configuration SharePoint was successful; everything worked, the portals are up and running, content is being populated, User Profile Service Synchronization is working, and the Search Service Application is up and running.

However, I was faced with a very challenging issue when crawling content. Crawling the SharePoint content source always returned a "timeout" error in the logs. Resolving this issue took a lot log monitoring, custom code to monitor the traffic, and long nights.

This means that the search crawl is sending an HTTP request to your portal, but it is not receiving an answer back. The authentication is fine, security is OK, but there is no HTTP trip back to the crawl server.

There are my suggestions to a Search Crawl Timeout issue; one of the following suggestions might resolve your issue:

  1. Make sure you disable the loopback on the crawler server. In my case, this did not help at all.
  2. CRL Check: Most DLL assemblies are digitally signed.  Each time signed assemblies are loaded, default system behaviour is to check with the owner of the root certificate that the cert with which the assembly was signed is still valid. SharePoint 2013 search checks few certificates, like crl.microsoft.com or *.akamaitechnologies.com. To resolve this issue, open the outbound internet connection . If this is not doable, then install the crl.microsoft.com certificate on the server, or add an entry to local server host file like this: 127.0.0.1 crl.microsoft.com. This way certificate checks does not need to validate the certificate over the internet;
  3. Add exceptions on the firewall to allow traffic for the certificates; or
  4. Open Internet; or
  5. Revisit the firewall rules.

 

I suggest to first looking into the firewall rules again. 9 out of 1, it is the firewall that is doing funny things to block traffic between the servers. In my case, the security team were using Cisco Smart Care firewall, which is an advanced firewall and it does not only look at the ports' rules. You will have to create exception for applications, because it detects SharePoint and it automatically blocks it if SharePoint as an app is not listed as one of the trusted apps.

 

 

Missing Content Search Web Part

In SharePoint 2013, all the content can now be surfaced using search.  The Search driven  web parts have their own Querying Builder user Interface which makes it very easy to select, filter and display the data that you want. However, content Search Web Part is only available in SharePoint 2013 Enterprise Edition. If you are using the Enterprise CALs, then you should see the search driven web part in your web part gallery.

But, this is not always the case if you have played around with the licensing in the farm. SharePoint 2013 provides a new feature called SharePoint User License Enforcement (SPULE) that a lot of people may not be aware of.  SPULE means that we can have a mix of different licenses in a single farm.  What this means, is that Enterprise features can be made available to those who need it, and Standard features to others.  This can save an organization a substantial amount related to cost of Client Access Licenses.

If for some reason you ran this command line: Set-SPUserLicensing, this will actually disable all your search driven web parts. Note that by default, the SPULE is not enabled.

To get an overview of the SPULE in your farm, run this command: Get-SPUserLicensing. If true is returned, this means that the SPULE has been enabled on your farm.

What you need to do is to disable the SPULE, and the Search driven web parts will appear again. Run this command Disable-SPUserLicensing, and voila! Your web parts are back in the gallery!

Note: You can set the SPULE based on different AD groups, and you can set it for different type of licenses. This TechNet article will explain to you how you can manipulate different SPULE in your farm.

SharePoint 2013 Search with SharePoint 2010 Farm

 Many customers are excited about the new features that SharePoint 2013 brings to the table. Small or large organizations who have implemented any SharePoint implementation project size hesitate to upgrade for many reasons, but they want to take advantage of some the new features of 2013.

Recently, I ran into a similar situation where I am working with a client who prefers to stay on SharePoint 2010 for content and collaboration, but instead of implementing FAST for SharePoint 2010, they decided to use SharePoint 2013 for search.
As we all know, F4SP is part of the 2013 platform now and it is not a standalone product anymore. For this reason, my client’s vision was to better off use SharePoint 2013 for search, rather than F4SP then go through the headaches of migration F4SP to 2013 or any future release of the product.
The introduction of the Service Applications in SharePoint 2010, made life easier to implement scalable architecture and to create large multi-tenants farms, where you can share and publish service applications across different SharePoint farms. The same architecture is carried to 2013, and now we have the ability to publish service applications from 2013 to 2010, allowing customers to take advantage of some new features of the 2013 platform.
Note that 2010 can consume 2013 service applications and not the other way around.
Here is a list of the service applications that you can publish in 2013 and consume in 2010:
1.     User Profile Service
2.     Search Service
3.     Managed Metadata Service
4.     Business Connectivity Services
5.     Secure Store Service
In my case, I will be providing details on how to publish the Search Service Application in 2013, and consume it in 2010 using the Search Center.
First Step: You need to establish a trust relationship between the two farms:
1.    Export the Farm and STS certificates from the SharePoint 2010 farm:
$rootCertificate = (Get-SPCertificateAuthority).RootCertificate
$rootCertificate.Export(“Cert”) | Set-Content C:\Certificates\2010FarmRoot.cer -Encoding byte
$stsCertificate = (Get-SPSecurityTokenServiceConfig).LocalLoginProvider.SigningCertificate
$stsCertificate.Export(“Cert”) | Set-Content C:\Certificates\2010FarmSTS.cer -Encoding byte
 
2.    Export the Farm certificate from the SharePoint 2013 farm:
$rootCertificate = (Get-SPCertificateAuthority).RootCertificate
$rootCertificate.Export(“Cert”) | Set-Content C:\Certificates\2013FarmRoot.cer -Encoding byte
 
3.    Import the SharePoint 2013 certificate into the SharePoint 2010 farm:
$trustCertificate = Get-PfxCertificate C:\Certificates\2013FarmRoot.cer
New-SPTrustedRootAuthority “2013 Trust”-Certificate $trustCertificate
 
4.    Import the SharePoint 2010 into the SharePoint 2013 farm:
$trustCertificate = Get-PfxCertificate C:\Certificates\2010FarmRoot.cer
New-SPTrustedRootAuthority “2013 Trust” -Certificate $trustCertificate
$stsCertificate = Get-PfxCertificate C:\Certificates\2010FarmSTS.cer
New-SPTrustedServiceTokenIssuer “2013 Trust” -Certificate $stsCertificate
 
Second Step: You need to publish the Search Service Application and set the permissions:
1.    Go to Central Admin à Manage Service Applications
2.    Click on your Search Service Application
3.    Click Publish; make sure you select the checkbox next to “Publish this Service Application to other farms”
4.    From the SharePoint 2010 farm, run the following command to get the Farm ID:
$farmID= Get-SPFarm
$farmID.Id 

5.    From the SharePoint 2013 farm, run the following commands:
$security=Get-SPTopologyServiceApplication | Get-SPServiceApplicationSecurity
$claimprovider=(Get-SPClaimProvider System).ClaimProvider
$principal=New-SPClaimsPrincipal -ClaimType “http://schemas.microsoft.com/sharepoint/2009/08/claims/farmid” -ClaimProvider $claimprovider -ClaimValue [FarmID]
Grant-SPObjectSecurity -Identity $security -Principal $principal -Rights “Full Control”
Get-SPTopologyServiceApplication | Set-SPServiceApplicationSecurity -ObjectSecurity $security
 
6.    From the SharePoint 2013 SSA, give the SharePoint 2010 Farm ID “Full Control” permissions
From 2010, you can connect to the 2013 SSA by providing the 2013 SSA published servive URL. 
Now, go into your 2013 SSA, add a SharePoint 2010 content source and run a full crawl. Once the crawl is completed, you will be able to search the content using your 2010 Search Center.
Note: If you need to take advantage of the results preview feature, you will need to install and configure Office Web Apps 2013 against your SharePoint 2013 farm.
 

 

Create and Extend SharePoint 2013 Search with PowerShell

Create the Search Service Application

In the first section of this article, I am going to show how you can create SharePoint 2013 Search Service Application Using PowerShell. This list of commands will allow you to name your own database, instead having a GUID based database name for search.

The architecture and design of search in SharePoint 2013 have changed a bit. There are more added components and more flexibility for high availability search farm, allowing the farm to index more than 100 million items.

There are several steps involved in the creation of a Search Service Application and defining the Search Topology. The steps are:

  1. Creating the Search Service Application
  2. Creating the Search Service Application Proxy
  3. Creating the Search Components
  4. Creating the Index

 

Instead of using Central Admin, I will be showing PowerShell commands to create SSA:

# Define the variables

$SSADB = “SharePoint_Demo_SearchAdmin”

$SSAName = “Search Service Application”

$SVCAcct = “mcm\sp_search”

$SSI = get-spenterprisesearchserviceinstance -local

 #1. Start the search services for SSI

Start-SPEnterpriseSearchServiceInstance -Identity $SSI

 #2. Create the Application Pool

$AppPool = new-SPServiceApplicationPool -name $SSAName”-AppPool” -account $SVCAcct

 #3. Create the search application and set it to a variable

$SearchApp = New-SPEnterpriseSearchServiceApplication -Name $SSAName -applicationpool $AppPool -databaseserver SQL2012 -databasename $SSADB

 #4. Create search service application proxy

$SSAProxy = new-SPEnterpriseSearchServiceApplicationProxy -name $SSAName” Application Proxy” -Uri $SearchApp.Uri.AbsoluteURI

 #5. Provision Search Admin Component

Set-SPEnterpriseSearchAdministrationComponent -searchapplication $SearchApp -searchserviceinstance $SSI

 #6. Create the topology

$Topology = New-SPEnterpriseSearchTopology -SearchApplication $SearchApp

 #7. Assign server(s) to the topology

$hostApp1 = Get-SPEnterpriseSearchServiceInstance -Identity “SPWFE”

New-SPEnterpriseSearchAdminComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchCrawlComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchContentProcessingComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchAnalyticsProcessingComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchIndexComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1 –IndexPartition 0

 #8. Create the topology

$Topology | Set-SPEnterpriseSearchTopology

 Extend the Search Service Application

Once SSA is created, we will need to clone the topology to be able to extend to other servers in the farm. In this script, we will be replicating all the Search components onto two servers in the farm, also creating 2 indexes. Here are the steps:

#1. Extend the Search Topology:

$hostApp1 = Get-SPEnterpriseSearchServiceInstance -Identity “AppSearch1”

$hostApp2 = Get-SPEnterpriseSearchServiceInstance -Identity “AppSearch2”

Start-SPEnterpriseSearchServiceInstance -Identity $hostApp1

Start-SPEnterpriseSearchServiceInstance -Identity $hostApp2

 

#3. Keep running this command until the Status is Online:

Get-SPEnterpriseSearchServiceInstance -Identity $hostApp1

Get-SPEnterpriseSearchServiceInstance -Identity $hostApp2

 #4. Once the status is online, you can proceed with the following commands:

$ssa = Get-SPEnterpriseSearchServiceApplication

$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active

$newTopology = New-SPEnterpriseSearchTopology -SearchApplication $ssa

#Assign components to the hosts

New-SPEnterpriseSearchAdminComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchCrawlComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchContentProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchAnalyticsProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1

New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1 –IndexPartition 0

New-SPEnterpriseSearchAdminComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2

New-SPEnterpriseSearchCrawlComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2

New-SPEnterpriseSearchContentProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2

New-SPEnterpriseSearchAnalyticsProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2

New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2

#Below is creating another index on host 2. If you want to replicate the index to the second server, then you don’t need this step.

New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2 –IndexPartition 1

 #5. Activate the topology:

Set-SPEnterpriseSearchTopology -Identity $newTopology

 

The above scenario is creating a search topology over 2 server farm. For larger search topology, you can just add more hosts to the topology and select which components to run on them.