Generated Knowledge Prompting vs Knowledge based-Tuning in the LLM Context

Pretrained Large Language Models LLMs are increasingly used and are known with their astonishing performances. While LLMs is continuously improving, it’s still limited in specific domains like mathematics and medicine. Various research has been dedicated to equipping LLMs with external tools and techniques providing more flexibility and enhancing the capabilities of large language models.

In this post, I’m excited to present Generated Knowledge Prompting (GKP) and Knowledge-based Tuning techniques. They are distinct in their approaches, these methods share the common goal of refining model performance and expanding its comprehension across various domains.

Generated Knowledge Prompting (GKP)

GKP is a prompting technique proposed by Liu et al. The key insight behind Generated Knowledge Prompting is to be able to generate useful knowledge from another language model or external sources, then provide the knowledge as an input prompt that is concatenated with a question.

The intention behind this technique is to steer the model’s learning process towards specific types of outputs. By crafting prompts that target particular knowledge domains or concepts, GKP effectively guides the model to understand and generate relevant information.

Consider a scenario where a language model needs to grasp medical terminology and concepts. Through generated Knowledge Prompting, the first step is to generate the medical knowledge. The second is to integrate the knowledge as an input in the prompts. The model will analyze the prompt and use its underlying medical knowledge to generate text that addresses the query or provides information on the specified topic. This method harnesses the power of structured prompts to shape the model’s understanding and output generation capabilities.

Knowledge-Based Tuning

On the other hand, knowledge-based tuning involves fine-tuning a pre-trained model by integrating external data or information from knowledge sources. This process seeks to augment the model’s performance on specific tasks or domains by enriching its understanding with additional knowledge.

Let’s take an example of legal texts. If a language model requires proficiency in comprehending legal language, knowledge-based tuning can be employed. By exposing the model to a dataset comprising legal documents, the model can assimilate legal terminology and concepts, thereby enhancing its ability to interpret and generate responses in legal contexts.

Difference Between the Two Techniques

While both Generated Knowledge Prompt GPK and Knowledge-based Tuning contribute to enhancing language models, they differ in their methodologies and objectives. GKP focuses on guiding the model’s learning trajectory through tailored prompts. Knowledge is provided to the LLM as an input, while knowledge-based tuning modifies the model or its training process to incorporate external knowledge directly. This results in a fine-tuned model.

In summary, Generated Knowledge Prompting and knowledge-based tuning represent complementary approaches to empowering language models with deeper comprehension and proficiency across diverse subject matters.

That’s all for today. I will share more knowledge around LLM. So don’t miss coming posts 🎉. Follow and Join the conversation with your experiences/suggestion in the comments below 🌸

Pip Install OS Error Python package installation System Does not have Windows Long Path Support Enabled

 I have been trying to install the Transformer to work on deep learning project. I used the command, which I used many times before and it works perfectly.

But this time, I got the error Could not install packages due to an OSError: [WinError 2] The system cannot find the file specified

pip install transformers --users

 

ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: ‘C:\\Users\\me\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python312\\site-packages\\transformers\\models\\deprecated\\trajectory_transformer\\convert_trajectory_transformer_original_pytorch_checkpoint_to_pytorch.py’
HINT: This error might have occurred since this system does not have Windows Long Path support enabled. You can find information on how to enable this at https://pip.pypa.io/warnings/enable-long-paths

In order to fix the OSError, we need to make Windows 10 Accept File Paths Over 260 Characters.

So here the steps to follow in order to fix the error and be able to install the transformers package.

Step 1: Open Registry Editor

To do so, use the “Windows + R” and write “regedit”

Another option is to search for Registry Editor

Step 2:

In the registry editor, use the left sidebar and navigate to “CurrentControlSet\Control\FileSystem”

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem

Step 3:

On the right, find a value named “LongPathEnabled” and double-click it. If you don't see the value listed, you'll need to create it by right-clicking the “FileSystem” key, choosing New > DWORD (32-bit) Value, and then naming the new value “LongPathEnabled”.

Change the value from 0 to 1 and save your changes.

You can now close the registry and restart the computer. For my side, I just restarted my Visual Studio and was able to install successfully the transformer.

🎉Perfect, hope this helps you also and don’t forget to:

• Like the post 👏
• Follow me for more Tips

The VIP Lane for TCP/UDP Applications: Unveiling the Power of AWS Global Accelerator

 Imagine your application is a rockstar, performing sold-out shows worldwide! But instead of cheering fans, you have frustrated users dealing with lag and slow loading times. This is where AWS Global Accelerator comes in, your trusty roadie ensuring your app’s performance is smooth sailing across the globe.

Today, we’ll dive deep into AWS Global Accelerator key features, understand how it compares to CloudFront, and wrap it up with some pro tips to boost your AWS Certification journey. Buckle up and get ready to unleash the power of global application performance!

AWS Global Accelerator is a networking service that exceed traditional limitations by enhancing the performance of a wide range of applications that utilize the TCP or UDP protocol. This is achieved through a technique called “edge proxying.”

Imagine strategically positioned outposts along a global network. These outposts, known as edge locations, intercept incoming user requests. Global Accelerator then intelligently analyzes factors like user location, network conditions, and application health to determine the optimal endpoint within your AWS infrastructure (potentially across multiple regions).

You can easily get started with AWS Global Accelerator using these steps:

• Create an accelerator using AWS console. 2 Static IP addresses will be provisioned for you.
• Configure endpoints Groups. You choose one or more regional endpoint groups to associate to your accelerator’s listener by specifying the AWS Regions to which you want to distribute traffic. Your listener routes requests to the registered endpoints in this endpoint group. You can configure a traffic dial percentage for each endpoint group, which controls the amount of traffic that an endpoint group accepts.
• Register endpoint for endpoint groups: Assign regional resources (ELB, Elastic IP, EC2, NLB) in each endpoint group. You can also set up weight to choose how much traffic will reach each service.

Key Features of AWS Global Accelerator:

• Static IP Addresses: No more fumbling with complex regional addresses. Global Accelerator gives your app a permanent, recognizable stage presence.
• Global Network Routing: Think of it as a teleport for your data! Global Accelerator whisks user requests to the closest AWS location, ensuring the fastest possible connection.
• Instant Failover: Is one of your application’s servers having an off night? No worries! Global Accelerator seamlessly redirects traffic to healthy backups, keeping the show running smoothly.
• Traffic Dial: Need to control the flow of users for A/B testing or a new feature rollout? Global Accelerator’s handy traffic dial lets you adjust the audience size for a specific region, like dimming the lights before a special announcement!
• Weighted Traffic Distribution: Have multiple versions of your application across different regions? Global Accelerator acts like a spotlight operator, directing the right amount of users to each version based on your preferences.

Perfect, but wait. Isn’t it very close to AWS CloudFront. I got the same feeling and it’s a great question to ask.

Indeed AWS Global accelerator and AWS CloudFront are quite similar and they both use the edge location. But here few differences that will help you to decide which one to use:

• AWS CloudFront is a content delivery network CDN that improves performance for both cacheable content (such as images and videos) and dynamic content (such as API acceleration and dynamic site delivery). CloudFront is used with TCP protocol. It provide Lambda@Edge and CloudFront functions to intercept request and execute short code.
• AWS Global Accelerator is a networking service that is a good fit for non-HTTP use cases, such as gaming (UDP), IoT (MQTT), or Voice over IP. It’s good also for HTTP use cases that require Static IP addresses or deterministic. Global Accelerator doesn’t provide any caching feature. It is a good fit for fast regional failover.

AWS Certification Champs, Take Note !

• The Global accelerator is used with TCP and UDP traffic. It doesn’t provide caching, neither lambda function on the edge
• AWS global accelerator can be used to remove the hassle of managing a scaling number of IP addresses and related security.
• With Global Accelerator, you can add or remove endpoints in AWS Regions, run blue/green deployment, and do A/B testing without having to update the IP addresses in your client applications.
• AWS Global accelerator provides great acceleration for latency-sensitive applications
• It’s possible to dynamically route multiple users to specific endpoint IP and port behind the accelerator. User traffic can be routed to a specific Amazon EC2 IP in single or multiple AWS regions. An example is a multi-player game where multiple players are assigned to the same session. Another example is VoIP or social media app that assign multiple users to a specific media server to initiate voice or video call session.
• Global Accelerator creates a peering connection with your Amazon Virtual Private Cloud using private IP addresses, keeping connections to your internal Application Load Balancers or private EC2 instances off the public internet.
• AWS Global accelerator improves performance for VoIP, online gaming and IoT sensors apps.
• You can’t directly configure on-premises resources as endpoints for your static IP addresses, but you can configure a Network Load Balancer (NLB) in each AWS Region to address your on-premises endpoints. Then you can register the NLBs as endpoints in your AWS Global Accelerator configuration.

P.S. That’s all for today, don’t forget to share your thoughts and experience with AWS Global Accelerator in the comment below!

Dive into Delightful Message Queues with AWS SQS

 

 The heart of the AWS SQS show? It’s the messages, bustling with data like little digital bees! Each message finds a cozy home in a queue, waiting patiently to be processed by a single, dedicated subscriber.

Amazon SQS throws on its invisibility cloak for a little while to prevent other consumers from grabbing it too soon. This cloak, called a visibility timeout, ensures each message gets its moment to shine and be processed properly. By default, this cloak lasts for 30 seconds, but you can adjust it to fit your needs — from a lightning-fast 0 seconds to a leisurely 12 hours.

Once the subscriber finishes the job, the message bids farewell and departs the queue, leaving space for the next one in line.

But wait, there’s more to this queueing magic! SQS offers a nifty feature called long polling. Imagine a queue full of eager messages, but instead of constantly checking if there are new ones (which can be tiring!), Amazon SQS sends a response after it collects at least one available message, up to the maximum number of messages specified in the Receive Message request. Amazon SQS sends an empty response only if the polling wait time expires.

Speaking of smooth, let’s talk about retries. Sometimes, even the most dedicated subscribers might encounter hiccups. That’s where the maxReceiveCount comes in. You can use this setting to tell SQS how many times it should attempt to process a message before moving on. This limits the number of times Lambda will retry to process a failed execution. As SQS natively supports Dead Letter Queue DLQ, it’s possible to create it and configure SQS to transfer failed message to that Queue.

But messages aren’t all about the content, they can carry extra information too! Think of it like a little luggage tag with details like timestamps, location data, or even a digital signature. These message attributes travel alongside the message itself (separately from the body), providing valuable context for the recipient.

Delay queues add another layer of fun to the mix. When a message enters the SQS, it takes a pre-defined break before being consumed. This is perfect for situations where tasks need to happen at a specific time. It’s possible to set delay seconds on individual messages, rather than on an entire queue. To do so, use Message timers to allow Amazon SQS to use the message timer’s DelaySeconds value instead of the delay queue’s DelaySeconds value.

With these delightful features at your fingertips, AWS SQS can help you build resilient, efficient, and scalable applications. So, what are you waiting for? Start queuing up for some cloud-based magic!

Join the conversation and share your SQS experiences in the comments below!

Good To know:

• Message size restriction is 256KB
• The default waiting time of a message in SQS Queue is 4 days
• For Lambda function consuming from SQS, the best visibility timeout is 6x the function timeout. This gives time for retries with extra buffer.
• To support cross-regions or SQS active mode, it’s possible to use SNS that delivers messages to SQS Queue in different regions. This is useful to divide customers by location for access to the nearest region.

Understand the VPC endpoint like a Pro and get ready for the AWS certification

Imagine sending your data on a private jet ✈️ instead of a bus 🚌 through rush hour traffic 🚦. That’s the magic of VPC endpoint: a secure, high-speed lane connecting your AWS resource directly to specific services, all within the safe confines of the AS network.

This blog post is your-first class ticket to understanding these powerful tools and get ready for you AWS certification. Buckle up and we will explore the two types of VPC endpoints:

• Interface endpoints: Think of them as private tunnels, shipping data directly to the service without ever touching the public internet.
• Gateway endpoints: these acts as secure gateways, allowing communication with AWS S3 and DynamoDB.

 As explained before, a VPC endpoints lets you privately connect your VPC to supported AWS services and VPC endpoint services.

VPC endpoints are virtual devices. They are horizontally scaled, redundant, and highly available Amazon VPC components that allow communication between instances in an Amazon VPC and services without imposing availability risks or bandwidth constraints on network traffic.

Gateway endpoints

A gateway endpoint is a virtual connection within your VPC that allow resources in your VPC to directly access AWS S3 and DynamoDB. The gateway targets specific IP routes in an Amazon VPC route table, in the form of a prefix-list, used for traffic destined to Amazon DynamoDB or Amazon Simple Storage Service (Amazon S3).

When you create a gateway endpoint, you select the VPC route tables for the subnets you enable. A route with a prefix-list is automatically added to each selected route table. Those routes can’t be modified or deleted unless you disassociate the route table from the gateway or you delete the gateway itself.

There is no additional charges for using the gateway endpoint, which makes it preferable over interface endpoint in case your need access to S3 or DynamoDB. Nevertheless, Gateway endpoint doesn’t allow access from on-premise networks, from peered VPC in other regions and through transit gateway. For those use cases, you must use the Interface endpoint.

Here short tips when working with the gateway endpoints:

• A gateway endpoint is available only in the region you created it. Be sure it’s the same region as your S3 bucket.
• You can have routes to S3 and/or DynamoDB in the same route table. But you can’t have multiple routes to the same service in a single route table.
• The instances access S3 or DynamoDB service using its public endpoints. Their security groups must allow traffic to and from the services using the prefix_list_id as destination with the TCP protocol on port 443.
• The Network ACLs for the subnets for theses instances must also allow traffic to and from the services.

You need to consider the AWS specificities in traffic routing. AWs uses the most specific route that matches the traffic to determine how to route it (longest prefix match). In route tables with endpoint route this means:

• If the route that sends all internet traffic (0.0.0.0/0) to an internet gateway, the endpoint route takes precedence for traffic sent to S3 or DynamoDB in the current region.
• Traffic destined for S3 or DynamoDB in a different region goes to the internet gateway as prefix lists are specific to a Region.
• If there is a route that specifies the exact IP address range for the S3 or DynamoDB in the same region, that route takes precedence over the endpoint route.

Interface endpoints

Interface endpoints enable connectivity to services over AWS PrivateLink.

But wait, what is PrivateLink. It’s an amazing service that allows you to to securely connect your resources in an Amazon Virtual Private Cloud VPC to specific AWS resources, other VPCs, and even on-premise applications, without ever exposing your data in the public internet.

Here some benefit of the PrivateLink:

• AWS PrivateLink gives on-premises networks private access to AWS services through Direct Connect.
• You can make services available to other accounts and VPCs that are accessed securely as private endpoints.
• If you use AWS PrivateLink with a Network Load Balancer to route traffic to your service or application, clients can connect to any service you host.
• Services configured to support AWS PrivateLink can be offered as a subscription service through the AWS Marketplace.

The interface endpoint is powered by the AWS PrivateLink. When configuring an interface VPC endpoint, an elastic network interface (ENI) with a private IP address is deployed in your subnet. This ENI acts as an entry point for traffic destined to a supported service. Remember, attaching a Security Group is crucial for access control.

Provisioning an interface endpoint incurs hourly charges based on its uptime and data processing.

To optimize costs and simplify management, consider hosting interface endpoint in a centralized VPC. All the “spoke” VPCs can then leverage these centralized endpoints via Transit Gateway or VPC peering, eliminating the need for individual endpoints in each VPC. This approach known as Centralized VPC endpoint architecture pattern, minimizes costs and streamlines access control with centralized Security Groups.

P.S. Don’t forget to share your thoughts and experience with VPC endpoints in the comment below!