If you want to use the S1 pricing plan, and not have it charge when you are not using it, the only way of achieving that is by using automation. Fortunately, this is reasonably trivial to achieve.
If you look at this template it is pretty much all configured to deploy a website from Github to Azure on demand. If you configure it according to your needs you can have a new Azure website online with 2 minutes of running the script.
Then you would have another script that deleted it once you had finished. Doing it this way you wouldn't lose any functionality, and learn quite a bit about what is possible with Azure along the way.
If you put your hosting plan onto the free tier, you will stop being charged for it. However if you have things like deployment slots and certificates these will be deleted. The ability to turn services on and off, is more to do with being able to scale services (like if you need 50 servers for an hour you can easily do that).
What you can do to make your solution temporary is to create a deployment script, using PoweSshell or ARM templates, then you can deploy your solution for exactly as long as you need it and then delete it again when you don't. In this sense you can turn your services on and off whenever required.
To change the App Service Plan via PowerShell, you can run the following command:
Set-AzureRmAppServicePlan -ResourceGroupName $rg -Name $AppServicePlan -Tier Free