The following is an explanation of the 2 types of Authentication. This is well documented on the web!
Classic Mode: This is nothing but Windows Authentication. If any web application is created with Classic Mode Authentication then it cannot applicable to configure forms based authentication. That is basically it.
Claims Based: In SharePoint 2010 for a web application we can enable both windows AND forms authentication. In earlier implementation to do this, we have to create two web applications which has different zones and different authentication. But, with the new claims based authentication a single application can have the capability to configure both windows and forms under single URL. All this is possible because of the authentication framework is built on Microsoft Identify Foundation, and it uses “Geneva” framework to handle this authentication.
Convert from Classic Windows to Claims Based Authentication: To configure a Windows Authentication application to use Forms Based Authentication then you can convert from Classic Mode to Claims Based. But, there is no UI exist for doing this conversion. The only way around is through PowerShell.
WARNING! This cannot be UNDONE!
At the PS Prompt Type the following commands and press enter after each one.
$app = get-spwebapplication "http://<your site's URL>"
$app.useclaimsauthentication = "True"
$app.Update() Don't forget the brackets!
IMPORTANT: If an application is created using Claims Based Authentication and if you want to convert to Classic Mode then it is not possible either through the UI or through PowerShell.
So you want to know if that SharePoint 2010 Web Application is configured with Claims Based or Windows Authentication?
The place to do this is via Central Admin -> Application Management -> Manage Web Applications. Then click on the Web Application and click on Authentication Providers in the Ribbon under the Security Section. BUT you might find that this icon is disabled like this screen shot.
Not to worry as you can enable it using one of two methods. One is to disable UAC via Control Panel but that's not really something you want to do. The other option is to Run your Browser as an Administrator like shown here.
Now you will see that the icon in the ribbon is enabled as shown here.
Now simply click the icon and you'll see something like the following 2 screen shots.
If you want to CHANGE the authentication type then you will need to do that via Power Shell which I will explain in another post.