dinsdag 3 december 2013

Optimizing SQL Server performance for Microsoft SharePoint 2010

Please consult a SQL Server Database Administrator and a SharePoint Administrator and discuss the settings before making any adjustments in your environment(s) – I can only give advice and not be responsible for any wrong configurations – remember that every environment is different!

90% of all data of a Microsoft SharePoint 2010/2013 farm is placed in the SQL Server databases, and every transaction from the users, all SharePoint services and SharePoint timer-jobs are working with these databases at the same time. SharePoint is depends on the performance of the web frontend servers and SQL servers. If the users should have the best experience of SharePoint, you need to optimize the resources that are used by SharePoint.

On the web frontend servers the Internet Information Server (IIS) delivers the content to the users. CPU and especially memory is important here as SharePoint uses the server memory to cache the information requested by the users. Usually this is not an issue as IT administrators can monitor these resources and do as a normal performance evaluation of Microsoft servers.

Fewer IT administrators or Database Administrators has knowledge of how SharePoint use the SQL Server and which components to optimize for delivering the best SharePoint end user experience. This article is about giving a little insight on this subject.

Physical or virtual servers?
Running the Windows Server and SQL Server directly on the hardware itself will give you the optimal performance. You will not have the advantages of the virtual environments such as failover, backup/restore and flexibility for the IT administrators. In my experiences a good virtual setup is as good as a physical one – and is the most chosen of my customers.

The SQL Server instance
On the SQL server we have some general best practices to follow and this means that the database administrators must change some default values to make the SQL server, and thereby also the applications, to perform better. SQL servers that host SharePoint databases are no exception, but certain settings should be changed to give even more power to the SharePoint servers. These settings can also have influence on other databases in the same SQL instance so it is advised to have a separate SQL Server for SharePoint – or at least a separate instance on an existing SQL Server.

If you choose to run multiple instances of SQL Server on a single Windows Server you need to control your CPU usage, minimum + maximum memory usage and database placement on disks. If the load on the databases and the SharePoint servers is small to medium you will achieve okay performance. But still, try to separate the SQL Servers to multiple Windows Servers as the used network bandwidth also has an effect on performance.

Memory is also used by the SQL Server for caching data requested by the applications. Therefore you should have plenty memory allocated to the SQL Server for best performance. It can be hard to evaluate how much memory is required as the SQL Server adjust the memory used depending on how much memory is available.

The CPU is used to perform calculations, queries and sorting the data. CPU usage can easily be monitored and evaluated if more CPU power is needed. Multiple processors with multiple cores is highly recommended as the SQL Server can take advantage of these.

The default file location of database and log should be set according to your needs (see the section Disk and file locations)

SQL Server version
The SQL Server should be a SQL Server 2008 (64 bit) with SP1 and CU 5 (or later version) installed even though former versions this edition of SQL Server is optimized for 64-bit and Microsoft SharePoint 2010/2013 can take advantage of the new functionality giving you’re a better overall performance. SharePoint will detect the SQL Server version and adjust the queries accordingly to the features available.

Memory limits
Add as much memory as possible on the Windows Server – the SQL Server will automatically allocate and cache based on the amount of RAM for the SQL Server instance.

The SQL Server will control the memory usage according to predefined rules in the SQL Server dynamic memory management.

If you only have one SQL Server instance on the server there’s not need to do memory adjustments!

If other applications or instances of SQL Server run on the same Windows Operating system then you need to adjust every SQL Server instance memory limits. By default the SQL Server dynamic memory management will use all available RAM on the operating system and release memory of this is required. Microsoft states this athttp://msdn.microsoft.com/en-us/library/ms177455(v=sql.105).aspx

The minimum memory limit is set to a value that the SQL Server instance cannot go below if it passes this limit. This value should be set to avoid the server to avoid unnecessary release of memory resources that the serve

The maximum memory limit defines the maximum memory the SQL Server instance is allowed to use. If you have multiple instances on the same Windows Server then evaluate how much each SQL Server instance should be allowed to use and how much should be available to the operating system and other programs.

Disk and file locations
You have the possibility to set the default locations for the database and the database log files for each SQL Server instance on the Windows Server. It is advised to use separate physical disks as the requirements for database files and log files are different. Performance wise you should also evaluate the load of the different databases as some might require more resources than others. Log files requires good write performance and the database files most read and caching performance

It is advised to place non-SharePoint databases on separate physical drives because SharePoint is very database transaction intensive.

The optimal disk cluster size (is set during formatting the drive) is 64k. Using this recommended cluster size the server reads 64k at the time and can deliver larger chucks of data to the SQL Server.

For the best performance you can add additional transaction log files to separate disks (not disks you are using for database files) Create same sized files on separate disks for tempdb, Content and Search databases. Use the same number of files as physical processor socket on your SQL Server to take advantage of the CPU’s (e.g. 2 data files on a 2 socket machine, 4 data files on a 4 socket machine).

System databases and SharePoint
The Model database, “Model”, is a system database that defines how a new database is default created by administrators and SharePoint. Set the default size on the Model-database to what size the future databases should be. If is advised to set this to what size each of your content database is believed to be in 1½-2 years. You should set the size to one value before the SharePoint installation and change this to your future default content database size afterwards.

The default created size could differ from what settings you want on non-SharePoint databases and therefore it is advised to have a separate SQL Server or SQL Server instance for the SharePoint databases.

In my experience the growth rate is not adjusted on new databases with these are created by SharePoint – so please check the database file growth periodically

The Temp database, “Tempdb”, is also a system database and this is used in a special way by SharePoint. It is used only as a working area and do not contain data that is important. At every SQL Server reboot the Temp database is rebuild. We should set the recovery mode to SIMPLE as we do not need to recover any information in the tempdb database.

The Temp database should be placed on a very fast drive for best performance on sorting and filtering operations. Microsoft SharePoint views are never faster than the “tempdb” can deliver data.

Database settings
Microsoft SharePoint use different databases for different purposes. Some contain configuration data and others the actual content. Most databases are created during the installation of the SharePoint farm so you must have your SQL Server settings correct before installing the product if you want to avoid performance issues or reconfiguration.

To make sure that putting content in your SharePoint databases is fast I recommend you to pre-grow these. Set the Database and log-sizes to what you expect within a year. This will also help reducing fragmentation and performance impact.

You should evaluate the usage of every database and adjust the
Database and log files
Location (fast separate drives)
Size (~ what you expect within 1 years)
Growth (~ 25% of the db size)
Recovery Model (FULL recommended)
Auto-shink -> set to the default: OFF

You should set the recovery mode to FULL on every SharePoint database to make sure that your data can be restored properly if a disaster happens – remember to do database and log backups to avoid running out of disk space. Also consult a SQL Server Administrator to ensure this is set up correctly – and check the backups with test-restore so you know that everything is good (and the restore time is okay with you).

It is advised that the Search databases are placed on even different physical disks or even a separate SQL Server, as the crawler- and index service is performing many transactions during a crawl and index propagation.

A small side-note: If you database administrators creates new databases for you, they must create these with a collation of Latin1_General_CL_AS_KS_WS. During the installation of the SQL instance they could set this to the default for new databases. If databases is created from the SharePoint user Interface, this value is set automatically.

Shrinking the logfiles – when?
The database transaction logfiles seems to keep on growing – especially if you do not back these up. This is because every transaction is put in these and not “flushed” until you do a backup. When you perform a backup the file gets “empty” and is ready for new transactions. If your transaction logfiles is too big then you need to shrink these – but please leave some empty space and don’t shrink to the minimum size. When a file needs to grow it has a large impact on performance.

Network
As all data is sent over the network the bandwidth between all servers must be optimal. A 1Gbit network connection on the servers is advised at all times. Also be sure that devices can handle the load – such as network routers, switches and firewalls.

I made a little graphical overview also:



Source:http://jespermchristensen.wordpress.com/2013/03/25/optimizing-sql-server-performance-for-microsoft-sharepoint-20102013/

zaterdag 20 juli 2013

Installing Observium Monitoring Tool

Hi all, today another new blog from my side. My colleague Rob has a nice monitoring tool called Observium. So i thought lets check this out and lets install this on a brand new Debian server.

Photo-main

Observium was started true a kickstarter project.
http://www.kickstarter.com/projects/588464062/observium-alerting

A small introduction about Observium.

"

Introduction

Observium is an Open Source, auto-discovering network monitoring platform written in PHP which supports a wide range of devices and operating systems. We collect data and status via SNMP and an optional Agent and present the information in a useful-to-engineers manner.
To keep things simple to manage, we try to discover everything that can be graphed or monitored on a device automatically. You usually don't know you need to graph something until after the event or outage! We even try to automatically discover neighboring devices seen via CDP and LLDP tables or OSPF neighbor tables. 
We believe that one of the key purposes of an NMS is to help engineers understand their networks. One of the very first features implemented was the ability to visualize a device's place in the network based on the devices it connected directly to, and the devices its interfaces share sub-nets with. 

"

Ok now a little tutorial on how to install this brilliant package.
Install the following packages:
aptitude install libapache2-mod-php5 php5-cli php5-mysql php5-gd php5-snmp \
php-pear snmp graphviz subversion mysql-server mysql-client rrdtool \
fping imagemagick whois mtr-tiny nmap ipmitool
  

Create and copy the following directories and files
$ mkdir /opt/observium/
$ cd /opt/observium/
$ cp config.php.default config.php
$ mkdir graphs rrd
$ chown www-data.www-data graphs rrd
$ mkdir /opt/observium/logs
  

  
CREATE DATABASE observium;
GRANT ALL PRIVILEGES ON observium.* TO 'observium'@'localhost'
IDENTIFIED BY '<observium db yourpassword>';
  
  

We need to create some Mysql database lines by following up the commands bellow
  
$ mysql -uroot -p
Enter password: <mysql root password>
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 238145

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE DATABASE observium;
Query OK, 1 row affected (0.10 sec)

mysql> GRANT ALL PRIVILEGES ON observium.* TO 'observium'@'localhost' IDENTIFIED BY 'observium';
Query OK, 0 rows affected (0.06 sec)
  
  

Now we need to add the config.php
cd/opt/observium/ nano config.php

### Database config
$config['db_host'] = "localhost";
$config['db_user'] = "observium";
$config['db_pass'] = "observium";
$config['db_name'] = "observium";

  

Ok now we need to add a virtual host to our apache
$ sudo vim /etc/apache2/sites-enabled/010-observium

<VirtualHost *:80>
       ServerName  observium.yourdomain.com
       DocumentRoot /opt/observium/html
       <Directory />
               Options FollowSymLinks
               AllowOverride None
       
       ErrorLog /var/log/apache2/error.log
       LogLevel warn
       CustomLog /var/log/apache2/access.log combined
       
</VirtualHost>
  
  

$ a2enmod rewrite
Enabling module rewrite.
Run '/etc/init.d/apache2 restart' to activate new configuration!

$ a2ensite observium
Enabling site observium.
Run '/etc/init.d/apache2 reload' to activate new configuration!

$ service apache2 restart
  

  
$ ./adduser.php
Add User Tool
Usage: ./adduser.php <username> <password> <level 1-10> [email]

$ ./adduser.php admin adminpass1 10 

$ ./adduser.php admin1 adminpass2 1 


The only thing left is to add a new record in your hostfile.

192.168.0.20 observium.yourdomain.com

When you have don this you can access the web page true IE by typing observium.yourdomain.com


Good luck installing :)

P.S
Make sure you have a router with SNMP.








vrijdag 12 juli 2013

Starting a basic (next-next-finish) SQL server installation for Sharepoint 2010

First of all you should install the following applications:

-Server 2008R2
-SQL 2008 R2
-Filterpack 2.0 64bit
-MSchart
-SYNC Setup
-Windows6.1 KB974405-X64

You also need to add the role IIS and add the feature .NET Framework 3.5.

So now we can start installing SQL server 2008 R2.

Step 1. Run Setup.exe from the SQL Server 2008 R2 install CD 
Step 2. Click on Installation on the left hand column
Step 3. Click on New SQL Server stand-alone or add features to an existing installation

Step 4. Click on Show Details and make sure that all the setup support rules have passed and then click OK
image
Step 5. Give in your Licence key
Step 6. Tick I accept the license terms and click Next
image
Step 7. Click Install
image
Step 8. Click Next
image
Step 9. Select the components you want to install and click Next
image
Step 10. Configure the Instance name and details and click Next
image
Step 11. Review the disk space requirements and click Next
image
Step 12. Configured the Service accounts that you want to use and click Next
image
Step 13. Add a SQL Server Administrator and Click Next
image
Make sure you Give in the right collation for SharePoint 2010: Server Collation SQL_Latin1_General_CP1_CI_AS 

Step 14. Click Next
image
Step 15. Click on Show Details and review the install rules and then click Next
image
Step 16. Do a final review of the install and click Install
image

Step 17. Click Next
image
Click 18. Click Close
image
Now you really need to install Service Pack 1

Step 19. Click Next
image
Step 20. Click I accept the license terms and then click Next
image
Step 21. Review the features that are going to be upgraded and click Next
image
Step 22. Click Next
image
Step 23. Click Update
image
Step 24. Click Next
image
Step 25. Click Close

For configurations tips and tricks check my SQL optimization blog

woensdag 10 juli 2013

Configuring Debian DNS server Advanced.


1. bind9 nameserver installation

Unless you prefer to install bind from a source code the installation is rather simple. On a Debian or Ubuntu Linux server you can install a bind nameserver with a following command:
apt-get install bind9 dnsutils
CentOS or Fedora alternative:
yum install bind dnsutils
Here we go good luck.

2. Creating a DNS zone file

At this stage we will need to create a file for a domain jeromic.nl  Navigate to /etc/bind/ directory execute following sequence of commands to navigate to etc/bind/zones/master/ if you dont have it create it like below
cd /etc/bind
mkdir -p zones/master
cd zones/master/
/etc/bind/zones/master directory will contain a zone file for a jeromic.nl domain. If you prefer to use another directory to hold this file you are free to do so. The following zone file db.jeromic.nlwill hold a DNS record to assist a nameserver resolve a fully qualified domain name to an IP address. Create and save db.jeromic.nl with a following content: you can create a site by typing cd /etc/bind/zones/master/ nano. Then put it the command below with you own domain. and save it as db.domain.extension.
;
; BIND data file for jeromic.nl

;
$TTL    3h
@       IN      SOA     ns1.jeromic.nl. admin.jeromic.nl. (
                          1        ; Serial
                          3h       ; Refresh after 3 hours
                          1h       ; Retry after 1 hour
                          1w       ; Expire after 1 week
                          1h )     ; Negative caching TTL of 1 day
;
@       IN      NS      ns1.jeromic.nl.
@       IN      NS      ns2.jeromic.nl.



jeromic.nl.    IN      A       192.168.0.10
ns1                     IN      A       192.168.0.10
blog.jeromic.nl         IN      CNAME   superitconsultant.blogspot.nl.

Here is just a quick overview of the records:
  • SOA Record: nameserver authoritative for a zone jeromic.nl is ns1.jeromic.nl and admin.jeromic.nl is an email address of a person responsible for this DNS zone.
  • NS Records: two nameservers for a jeromic.nl zone are ns[1,2].jeromic.nl
  • MX ( Mail Exchange): jeromic.nl mail exachange record. Number 10 means a preference for discarting a records A : A simply means address inanother words in jeromic.nl zone a ns1 would ahve a A ( address ) 192.168.0.10.
  • CNAME Record ( Canonical Name record ): restart the query using the canonical name instead of the original name

3. address-to-name mappings

At this stage the bind DNS server can resolve an IP address mapped to a jeromic.nl host. What we should do now is the teach our nameserver the other way around, which is, to resolve a host from an IP address. For this we are going to need yet another file and that is db.192.168.0 with a following content:
PTR
;
; BIND reverse data file for 0.168.192.in-addr.arpa
;
$TTL    604800
0.168.192.in-addr.arpa.      IN      SOA     ns1.jeromic.nl. admin.jeromic.nl. (
                          1         ; Serial
                          3h       ; Refresh after 3 hours
                          1h       ; Retry after 1 hour
                          1w       ; Expire after 1 week
                          1h )     ; Negative caching TTL of 1 day
;
0.168.192.in-addr.arpa.       IN      NS      ns1.jeromic.nl.
0.168.192.in-addr.arpa.       IN      NS      ns2.jeromic.nl.

10.0.168.192.in-addr.arpa.   IN      PTR     jeromic.nl.
  • PTR: a NDS record used for a mapping of an IP address to a host name.

4. Updating a BIND Configuration File

At this point we should have two files ready:
  • /etc/bind/zones/master/db.jeromic.nl
  • /etc/bind/zones/master/db.192.168.0
All we need to do now is to insert both zone file names into a bind's configuration file named.conf.local. To do that add following lines into this file:
zone "jeromic.nl" {
       type master;
       file "/etc/bind/zones/master/db.jeromic.nl";
};

zone "0.168.192.in-addr.arpa" {
       type master;
       file "/etc/bind/zones/master/db.192.168.0";
};
Last thing before we go ahead to check a configuration is to add and IP address af a stable DNS server to a named.conf.options file. This IP address is used in case that a local DNS server do not know the answer the a name resolution query. In IP address of a DNS server in many cases is provided by your Internet provider. Alternatively if you are google fan use 8.8.8.8.
Replace a following blog of text withing a named.conf.options file:
       // forwarders {
       //      0.0.0.0;
       // };
with new stable DNS server IP address
        forwarders {
              8.8.4.4;
         };
Ok now you are finished. You can check DNS by doing a NSlookup
root@debian:~# nslookup
> 192.168.0.10
Server:         192.168.0.10
Address:        192.168.0.10#53

** server can't find 10.0.168.192.in-addr.arpa.: NXDOMAIN
> blog.jeromic.nl
Server:         192.168.0.10
Address:        192.168.0.10#53

Name:   blog.jeromic.nl
Address: private ip
Thank and please leave a message or comment :)

dinsdag 9 juli 2013

Site Redirection in Debian

Hi all, I finally managed to redirect a site in Debian great thanks to my brilliant colleague Rob Maas (Check out his blog http://www.progob.nl/robmaaseu/ ).

First of all we tried to forward this site to my local domain within Debian. We did this thru the zone file in Bind9.

$TTL    604800
@       IN      SOA     NS1.domain.nldomain.thuis.domain.nl. (
                2013070901              ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                         604800 )       ; Negative Cache TTL
;
;Nameservers
;
Jeromic.nl      IN      NS      NS1.domain.nl

@       IN      NS      domain.nl.
@       IN      A       127.0.0.1
@       IN      AAAA    ::1

;
;Reverse Proxy
*                       IN      A       [my private ip]

;
;Alias
blog.jeromic.nl         IN      CNAME   superitconsultant.blogspot.nl

Only that didn't work. so Rob gave me another option to do this in apache.

cd /etc/apache2/sites-enabled/ nano 014-blog

After this we add the following in our Virtual host file.

<VirtualHost *:80>
       ServerName blog.domain.nl

       CustomLog       /var/log/apache2/blog_access.log combined
       ErrorLog        /var/log/apache2/blog_error.log

       Redirect / http://http://superitconsultant.blogspot.nl/

</VirtualHost>

After this we restarted the service

service apache2 restart

And the redirection is working properly

All thanks go to Rob Maas.

Thanks and please leave a comment

maandag 8 juli 2013

List of all Sharepoint Service applications with descriptions

Hi, today i'm working on a sheet with all the descriptions of all SharePoint Services which are integrated or a option for Sharepoint 2010. The plan is to get lots of SMART information. and put all of these on a letter format paper. With this information we can create our own cheat sheet.


ServiceDescription
Access Services
Makes users capable to view access databases inside of the web browser
Application Registry Service
Makes users capable to work with business data 
Business Data Catalog
Makes it possible to Embed a business application into SharePoint 2010
Excel Services
Makes users capable to view excel sheets inside of the web browser
Lotus Notes Connector
Makes users capable to Find Lotus Domino/Notes data
Managed Metadata Service
This will make sure that Teams and departments can create their own hierarchy keyword and social tags. So the can publish this thru the whole organisation
People
Makes it possible to import profiles from AD which will contain the information for MY site. Besides this you are able to search to people 
PerformancePoint Services
Gives the opportunities for PerformancePoint Services.
Search Service Application
Makes search queries and indexing possible inside content.
Secure Store Service
Makes it possible to secure data and save these with restrictions
State Service
Makes it possible to determine a status of a service or component. So when the will been load balanced these will not get lost
Subscription Service
This service will be used in Multi-Tenant Mode (hosting)
Usage and Health data collection 
Collects status data from the SharePoint farm
User Profile Service 
Offers profile services. 
Visio Graphics Service
Makes it possible to Show Visio diagrams in the web browser, and refine these dynamically
Web Analytics Web Service
Makes it possible to create analytics reports of the site. (how many users, how many page views etc) 
Word Conversion Service Application
Makes it possible to convert large numbers of documents.

Additional Services:


Service
Description
Office Web Applications
Offers the opportunity to provide several office applications in the webbrowser. the next applications will be provided Word 2010, Excel 2010, PowerPoint 2010 en OneNote 2010.
FAST Search Server
FAST Search for SharePoint 2010 is an advanced search in SharePoint 2010
Microsoft Project Server 2010
Project Server 2010 is the follow up for Project Server 2007 and Project Portfolio Server 2007. this offers a complete package for Enterprise Project Management (EPM) and Project Portfolio Management (PPM) processes.

If you have any other option or suggestion, or maybe i forgot something please let me know.

Regarding the home server Lab i'm running into some trouble. I need to redirect a site in my Linux Debian DNS so i added the cname to my zone only this doesn't work for now. 
This evening i will continue configuring my home Domain Controller. And possibly i will start with building up the first server for Sharepoint.

Please leave a comment.