Jump to content

Recommended Posts

Presentation
Originaly developped by Jeff Starr, Blackhole is a security plugin which trap bad bots, crawlers and spiders in a virtual black hole.
Once the bots (or any virtual user!) visit the black hole page, they are blocked and denied access for your entire site.
This helps to keep nonsense spammers, scrapers, scanners, and other malicious hacking tools away from your site, so you can save precious server resources and bandwith for your good visitors.

Β 

How It Works
You add a rule to your robots.txt that instructs bots to stay away. Good bots will obey the rule, but bad bots will ignore it and follow the link... right into the black hole trap. Once trapped, bad bots are blocked and denied access to your entire site.


The main benefits of Blackhole include:

Quote

Stops leeches, scanners, and spammers
Saves server resources for humans and good bots
Improves traffic quality and overall site security

Β Bots have one chance to obey your site’s robots.txt rules. Failure to comply results in immediate banishment.

Β 

Features

  • Disable Blackhole for logged in users
  • Optionally redirect all logged-in users
  • Send alert email message
  • Customize email message
  • Choose a custom warning message for bad bots
  • Show a WHOIS Lookup informations
  • Choose a custom blocked message for bad bots
  • Choose a custom HTTP Status Code for blocked bots
  • Choose which bots are whitelisted or not

Β 
Instructions

  1. Install the module
  2. Create a new page and assign to this page the template "blackhole"
  3. Create a new template file "blackhole.php" and call the module $modules->get('Blackhole')->blackhole();
  4. Add the rule to your robot.txt
  5. Call the module from your home.php template $modules->get('Blackhole')->blackhole();

Β Bye bye bad bots!


Downloads

Β 

Screen

blackhole.gif.8360604767dfcff7430cf4d317a11b94.gif

Β 


Β Enjoy :neckbeard:

Edited by flydev
module directory link
  • Like 15
Link to post
Share on other sites

Nice module, thanks for sharing.

I wonder though how effective it really is reading the last two sections "caveat emptor" and "blackhole whitelist":

https://perishablepress.com/blackhole-bad-bots/#blackhole-whitelist

Quote

Whitelisting these user agents ensures that anything claiming to be a major search engine is allowed open access. The downside is that user-agent strings are easily spoofed, so a bad bot could crawl along and say, β€œHey look, I’m teh Googlebot!” and the whitelist would grant access. It is possible to verify the true identity of each bot, but doing so consumes significant resources and could overload the server. Avoiding that scenario, the Blackhole errs on the side of caution: it’s better to allow a few spoofs than to block any of the major search engines.

Β 

  • Like 3
Link to post
Share on other sites

Its a shared host.

9 hours ago, flydev said:

what say <?php echo ini_get('allow_url_fopen'); ?> ?

It says true.

For the moment I have disabled this module because the loading time of the page increases significantly.

Link to post
Share on other sites
  • 1 month later...

I have installed it again but now I have only included the module in the blackhole.php (not on the home or other page) only to see if it works. It works now, but the loading time of the page is approx. 21 seconds!!!!

I have added a hidden link in my site to the blackhole.php and if I click on it my IP will be stored in the DAT file - works well. In the mail that I got afterwards there was a hint about a Port problem:

Whois Lookup:

Timed-out connecting to $server (port 43).

I am on a shared host so it seems that this port is not free. The strange thing is that I have disabled the Who is Lookup in my settings of the module

Screenshot(8).png.9ef7ce303425b5bc8ff54ec9cdf2ba76.png

Best regards JΓΌrgen

  • Thanks 1
Link to post
Share on other sites

Thanks you @Juergen .

About the port 43, its common that this port is blocked by default and - depending on the hosting provider - can be configured trough the panel provided.

59 minutes ago, Juergen said:

The strange thing is that I have disabled the Who is Lookup in my settings of the module

Will look at it this afternoon as I am deploying this module a on a production site. Stay tuned, thanks again mate.

  • Like 1
Link to post
Share on other sites

Works like a charm now! Would be great if the hard coded url of the "contact the administrator" page could be selected out of PW pages.

Thanks for the update!!!

Edit: It would be better if you add multilanguage support to the custom message textareas :)

Β 

Β 

Edited by Juergen
  • Like 1
Link to post
Share on other sites
3 hours ago, flydev said:

I will try to do it, I never played with modules and multilanguage

Its not so important, because only bad bots will see it and probably no humans (I hope so). By the way 2 bots from China were caught in the trap - works!!!:)

  • Like 1
  • Haha 1
Link to post
Share on other sites

Good and funny !

Β 

13 hours ago, Juergen said:

because only bad bots will see it and probably no humans

For example, on the site I deployed the module, it is a custom dashboard with sensible informations, I had to take care of hand crafted request which could retrieve data from other users. When this behavior is detected, the user is logged out, the role login-disabled is assigned and then the user is redirected into the blackhole to be banned.

Β 

public function SecureParks() {
        if($this->input->post->park) {
            $ids = explode('-', $this->sanitizer->pageName($this->input->post->park));
            $userroles = $this->getParkRoles();
            $userhaveright = $this->searchForParkId($ids[2], $userroles);
            if ($userhaveright === null) {
                $this->user->addRole('login-disabled');
                $this->user->save();
                $this->session->logout();
                $this->session->redirect($this->pages->get('/blackhole/')->url); // :)
            }
        }
    }

Β 

  • Like 2
Link to post
Share on other sites
  • 2 weeks later...

Just a thought:

I think it would be nice to store the banned IPs also in a logfile, so you have them in one place with the other protocols.

Fe:

$log->save('blackhole', 'Banned IP')

You can also add fe a checkbox in the module settings to offer enabling and disabling of this feature.

What do you think? Might this be useful for others too?

  • Like 2
Link to post
Share on other sites

I was also thinking to add a new feature from where we could monitor 302/404 HTTP code and redirect the "guest" into the blackhole.

For example, all those try :

  • /phpMyAdmin/scripts/_setup.php
  • /w00tw00t.at.ISC.SANS.DFind:)
  • /blog/wp-login.php
  • /wp-login.php
  • etc.

will be banned.

I still don't know if I code all the feature or if I should hook into Jumplinks from @Mike Rockett.

Link to post
Share on other sites
3 minutes ago, flydev said:
  • /blog/wp-login.php
  • /wp-login.php

I also have a lot of these requests in my 404 logger protocol :(.

I think if there is module that can handle it - use it.Β  Check if the module is installed first. If not output a message that this feature is only available if Jumplinks is installed.

I dont have Jumplinks installed and I dont know how well it works, but before starting to code from the beginning I would try to use an existing solution first.

Link to post
Share on other sites

I use Redirect gone ... in .htaccess

Redirect gone /wp-login.php

for all that stuff. (First I log 404s for a period, than I add those candidates to the .htaccess, before ProcessWires entries!!)

I think it is better to not invoke PW for this stuff, (lesser overhead on the server!), instead use apache custom error page(s).

410_wp-login_php.thumb.jpg.86905f9ab46e4529163d4bc51d3df7e3.jpg

47ms is fast! :)

Β 

PS: 410 is better than 404, as I also use this for SearchEngineRequests that try to reach URLs that do not exist since 10 years or so. Normally the SEs should flush their cache on 410 returns.

Edited by horst
  • Like 2
  • Thanks 1
Link to post
Share on other sites

Ok guys, I get what you mean, so what about a module with this flow ?

  1. monitor and log HTTP error code for a period
  2. if an entry / request is superior of N then
  3. backup .htaccess file (versioning it)
  4. add new entries to the .htaccess file

Β 

Does it make sense or I should let the user manage their .htaccess file manually with a FAQ or something ?

  • Like 2
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By FireWire
      Hello community!

      I want to share a new module I've been working on that I think could be a big boost for multi-language ProcessWire sites.

      Some background, I was looking for a way for our company website to be efficiently translated as working with human translators was pretty laborious and a lack of updating content created a divergence between languages. I, and several other devs here, have talked about translation integrations and have recognized the power that DeepL has. DeepL is an AI deep learning powered service that delivers translation quality beyond any automated service available. After access to the API was opened up to the US, I built Fluency, a DeepL translation integration for ProcessWire.
      Fluency brings automated translation to every multi-language field in the admin, and also provides a translation tool allowing the user to translate their text to any language without it being inside a template's field. With Fluency you can:
      Translate any plain textarea or text input Translate any CKEditor content (yes, with markup) Translate page names for fully localized URLs on every page Translate your in-template translation function wrapped strings Translate modules Fluency is free, and now so is DeepL
      Since this module was first built DeepL has introduced free Developer accounts that allow anyone to start using Fluency at zero cost and beginning with the version 0.3.0 release Fluency now supports free DeepL accounts. As of June 2021 DeepL supports translation to 26 languages and continues to offer more!
      Installation and usage is completely plug and play. Whether you're building a new multi-language site, need to update a site to multi-language, or simply want to stop manually translating a site and make any language a one-click deal, it could not be easier to do it. Fluency works by having you match the languages configured in ProcessWIre to DeepL's. You can have your site translating to any or all of the languages DeepL translates to in minutes (quite literally).
      Let's break out the screenshots...
      When the default language tab is shown, a message is displayed to let users know that translation is available. Clicking on each tab shows a link that says "Translate from English". Clicking it shows an animated overlay with the word "Translating..." cycling through each language and a light gradient shift. Have a CKEditor field? All good. Fluency will translated it and use DeepL's ability to translate text within HTML tags. CKEditor fields can be translated as easily and accurately as text/textarea fields.

      Repeaters and AJAX created fields also have translation enabled thanks to a JavaScript MutationObserver that searches for multi-language fields and adds translation as they're inserted into the DOM. If there's a multi-language field on the page, it will have translation added.

      Same goes for image description fields. Multi-language SEO friendly images are good to go.

      Creating a new page from one of your templates? Translate your title, and also translate your page name for native language URLs. (Not available for Russian, Chinese, or Japanese languages due to URL limitations). These can be changed in the "Settings" tab for any page as well so whether you're translating new pages or existing pages, you control the URLs everywhere.

      Language configuration pages are no different. Translate the names of your languages and search for both Site Translation Files (including all of your modules)

      Translate all of the static text in your templates as well. Notice that the placeholders are retained. DeepL is pretty good at recognizing and keeping non-translatable strings like that. If it is changed, it's easy to fix manually.

      Fluency adds a "Translate" item to the CMS header. When clicked this opens up a modal with a full translation tool that lets the user translate any language to any language. No need to leave the admin if you need to translate content from a secondary language back to the default ProcessWire language. There is also a button to get the current API usage statistics. DeepL account owners can set billing limitations via character count to control costs. This may help larger sites or sites being retrofitted keep an eye on their usage. Fluency can be used by users having roles given the fluency-translate permission.

      It couldn't be easier to add Fluency to your new or existing website. Simply add your API key and you're shown what languages are currently available for translation from/to as provided by DeepL. This list and all configuration options are taken live from the API so when DeepL releases new languages you can add them to your site without any work. No module updates, just an easy configuration. Just match the language you configured in ProcessWire to the DeepL language you want it to be associated with and you're done. Fluency also allows you to create a list of words/phrases that will not be translated which can prevent items such as brands and company names from being translated when they shouldn't

      Β 
      Limitations:
      No "translate page" - Translating multiple fields can be done by clicking multiple translation links on multiple fields at once but engineering a "one click page translate" is not feasible from a user experience standpoint. The time it takes to translate one field can be a second or two, but cumulatively that may take much longer (CKEditor fields are slower than plain text fields). There may be a workaround in the future but it isn't currently on the roadmap. No "translate site" - Same thing goes for translating an entire website at once. It would be great, but it would be a very intense process and take a very (very) long time. There may be a workaround in the future but it isn't on the roadmap. No current support for Inline CKEditor fields - Handling for CKEditor on-demand hasn't been implemented yet, this is planned for a future release though and can be done. I just forgot about it because I've never really used that feature personally.. Alpha release - This module is in alpha. Releases should be stable and usable, but there may be edge case issues. Test the module thoroughly and please report any bugs via a Github issue on the repository or respond here. Please note that the browser plugin for Grammarly conflicts with Fluency (as it does with many web applications). To address this issue it is recommended that you disable Grammarly when using Fluency, or open the admin to edit pages in a private window where Grammarly may not be loaded. This is an issue that may not have a resolution as creating a workaround may not be possible. If you have insight as to how this may be solved please visit the Github page and file a bugfix ticket.
      Requirements:
      ProcessWireΒ  3.0+ UIKit Admin Theme That's Fluency in a nutshell. A core effort in this module is to create it so that there is nothing DeepL related hard-coded in that would require updating it when DeepL offers new languages. I would like this to be a future-friendly module that doesn't require developer work to keep it up-to-date.
      The Module Is Free
      This is my first real module and I want to give it back to the community as thanks. This is the best CMS I've worked with (thank you Ryan & contributors) and a great community (thank you dear reader).
      DeepL Developer Accounts
      In addition to paid Pro Developer accounts, DeepL now offers no-cost free accounts. Now all ProcessWire developers and users can use Fluency at no cost.
      Learn more about free and paid accounts by visiting the DeepL website. Sign up for a Developer account, get an API key, and start using Fluency today.
      Download & Feedback
      Download the latest version here
      https://github.com/SkyLundy/Fluency-Translation/archive/main.zip
      Github repository:
      https://github.com/SkyLundy/Fluency-Translation
      File issues and feature requests here (your feedback and testing is greatly appreciated):
      https://github.com/SkyLundy/Fluency-Translation/issues
      Β 
      Thank you! Β‘Gracias! Ich danke Ihnen! Merci! Obrigado! Grazie! Dank u wel! DziΔ™kujΔ™! Бпасибо! γ‚γ‚ŠγŒγ¨γ†γ”γ–γ„γΎγ™οΌ 谒谒你!

    • By monollonom
      (once again I was surprised to see a work of mine pop up in the newsletter, this time without even listing the module on PW modules website πŸ˜…. Thx @teppo !)
      Github: https://github.com/romaincazier/FieldtypeQRCode
      Modules directory:Β https://processwire.com/modules/fieldtype-qrcode/
      This is a simple module I made so a client could quickly grab a QR Code of the page's url in the admin.
      There's not much to it for now, but if need be you can output anything using a hook:
      $wire->addHookAfter("FieldtypeQRCode::getQRText", function($event) { $event->return = "Your custom text"; }) You can also output the QR code on your front-end by calling the field:
      echo $page->qr_code_field; The module uses the PHP library QR Code Generator by Kazuhiko Arase. When looking for a way to generate a QR Code in PW I came across @ryan's integration in his TFA module. I'm not very familiar with fieldtype/inputfield module development so I blindly followed @bernhard (great) tutorial and his BaseFieldtypeRuntime. At some point I'll take a deeper look to make a module on my own.
      Some ideas for improvements :
      add the ability to choose what to ouput : page's url / editUrl / file(s) / image(s) / ... allow to output multiple QR codes ?
    • By Chris Bennett
      https://github.com/chrisbennett-Bene/AdminThemeTweaker
      Inspired by @bernhard's excellent work on the new customisable LESS CSS getting rolled into the core soon, I thought I would offer up the module for beta testing, if it is of interest to anyone.

      It takes a different approach to admin styling, basically using the Cascade part of CSS to over-ride default UiKit values.
      Values are stored in ModuleConfig Module creates a separate AdminThemeTweaker Folder at root, so it can link to AdminThemeTweaker.php as CSS AdminThemeTweaker.phpΒ reads the moduleΒ values, constructs the CSS variables then includes the CSS framework Can be switched on and off with a click. Uninstall removes everything, thanks to bernhard's wonderful remove dir & contents function.
      It won't touch your core. It won't care if stuff is upgraded.Β You won't need to compile anything and you don't need to touch CSS unless you want to.

      It won't do much at all apart from read some values from your module config, work out the right CSS variables to use (auto contrast based on selected backgrounds) and throw it on your screen.
      You can configure a lot of stuff, leave it as it comes (dark and curvy), change two main colors (background and content background) or delve deep to configure custom margins, height of mastheads,Β and all manner of silly stuffΒ I never use.

      Have been developing it for somewhere around 2 years now. It has been (and will continue to be) constantly tweaked over that time, as I click on something and find something else to do.
      That said,Β it is pretty solid and has been in constant use as my sole Admin styling option for all of thoseΒ 2 years.

      If nothing else, it would be great if it can provide any assistance toΒ @bernhardΒ or other contributor's who may be looking to solve some of the quirkier UiKit behavior.
      Has (in my opinion) more robust and predictableΒ handling of hidden Inputfields, data-colwidths and showIf wrappers.
      I am very keen to help out with that stuff inΒ any way I can, though LESS (and any css frameworks/tools basically) are not my go.
      I loveΒ CSS variables and banging-rocks-together, no-dependency CSS you can write with notepad.



      Β 

    • By opalepatrick
      I see old posts saying that repeaters are not the way to go in Custom Process Modules. If that is the case, when using forms (as I am trying to do) how would one tackle things like repeat contact fields where there can be multiple requirements for contact details with different parameters? (Like point of contact, director, etc) or even telephone numbers that have different uses?
      Just for background I am creating a process module that allows me to create typesΒ of financial applications in the admin area (no need to publish any of this, pure admin) that require a lot of personal or company information.
      Maybe I am thinking about this incorrectly?
    • By HMCB
      I ran acrossΒ a reference toΒ IftRunner module. The post was 6 years ago. I cant find it in available modules. Has it been pulled?
Γ—
Γ—
  • Create New...